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-J6.  Abstract 


'This  study  assessed  the  effectiveness  of  four  recurrent  training  courses  at  the 
FAA  Management  Training  School  (MTS).  The  courses  evaluated  were  Performance 
Improvement  and  Employee  Appraisal  (PIP/PER),  Labor  Relations  for  Management  (LMR), 
Constructive  Discipline  (CD),  and  Managerial  Effectiveness  (ME).  Questionnaires 
concerning  the  usefulness  of  course  content,  effects  on  supervisory  behavior,  and 
impact  on  the  organizational  unit  were  sent  to  randomly  selected  graduates  of 
these  courses.  Immediate  sifperiors  and  supervisees  of  these  graduates  were  also 
sent  questionnaires  concerning  the  effects  of  the  course  on  the  behavior  of  the 
graduates.  A total  of  402  graduates,  263  immediate  superiors,  and  280  supervisees 
responded  to  the  survey.  Approximately  90  percent  of  the  graduates  of  each  of  the 
courses  evaluated  the  training  as  useful  overall.  Self-reported  behavioral  changes 
were  noted  most  often  in  the  areas  of  increased  knowledge  and  understanding  of 
the  subject  under  consideration  in  the  course,  increased  ability  to  administer 
these  programs,  and  improved  supervisory  and  interpersonal  skills  in  general. 
Behavioral  ratings  by  graduates,  their  immediate  superiors,  and  supervisees 
generally  supported  the  conclusion  that  these  courses  had  a beneficial  impact  on 
supervisory  behavior.  V. 
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AN  EVALUATION  OF  FOUR  MTS  RECURRENT  TRAINING  COURSES 


PART  ONE.  An  Overview 

I.  Introduction. 

This  survey  was  undertaken  to  assess  the  effectiveness  of  the  recurrent 
training  courses  taughc  for  FAA  supervisors  and  managers  at  the  FAA  Management 
Training  School  (MTS).  The  four  courses  evaluated  were  the  Performance 
Improvement  and  Employee  Appraisal  (PIP/PER)  course,  the  Labor  Relations  for 
Management  (LMR)  course,  the.  Constructive  Discipline  (CD)  course,  and  the 
Managerial  Effectiveness  (ME)  course. 

Three  groups  of  FAA  personnel  were  surveyed  about  the  effectiveness  of 
each  course.  Graduates  of  each  course  were  asked  about  the  usefulness  of  the 
course,  how  the  course  had  influenced  their  supervisory  or  managerial  behavior, 
and  how  attendance  at  the  course  had  influenced  conditions  in  their  organiza- 
tional units.  The  imnediate  superiors  and  supervisees  of  graduates  were  asked 
to  what  extent  they  perceived  changes  in  the  behavior  of  graduates  of  the 
recurrent  course.  Supervisees  were  also  surveyed  about  conditions  within  their 
l units. 

The  judgments  of  these  three  groups  of  respondents  were  obtained  through 
detailed  questionnaires  designed  for  each  course  (Appendixes  I,  II,  and  III). 
These  questionnaires  were  sent  out  to  individuals  according  to  a stratified 
random  sampling  procedure  based  on  the  distribution  of  the  graduates  of  these 
courses  across  the  various  FAA  services  and  regions.  A total  of  945  usable 
questionnaires  were  returned,  including  those  from  402  graduates  of  the 
courses,  263  imnediate  superiors,  and  280  supervisees.  The  returns  of  gradu- 
ates were  sufficient  to  be  95  percent  confident  that  obtained  values  were  within 
plus  or  minus  10  percent  of  values  that  would  have  been  obtained  from  the 
entire  group  of  over  7,300  graduates  of  these  courses. 

The  findings  for  the  overall  course  evaluations  indicated  that  each  of  the 
courses  was  generally  viewed  as  a positive  experience  by  approximately  90 
percent  of  the  respondents. 

When  graduates  were  asked  how  the  course  had  been  most  helpful,  the  most 
frequent  response  for  each  course  related  to  increased  understanding  of,  and 
ability  to  apply,  the  concepts  of  the  course.  Personal  growth,  the  opportunity 
to  work  through  problems  with  others,  and  increased  awareness  of  one's  role  as 
a supervisor  or  manager  were  also  commonly  mentioned. 

I 
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Graduates  of  the  courses  showed  little  unanimity  in  describing  the  areas 
of  the  courses  that  need  improvement.  While  comments  about  some  aspect  of 
course  content  were  most  frequent,  specific  suggestions  for  changes  in 
subject  matter  were  quite  varied  in  their  focus  for  each  of  the  courses. 

About  the  only  specific  theme  that  recurred  with  any  frequency  was  the  request 
for  a more  practical  emphasis  in  the  teaching. 

Assessments  of  the  usefulness  of  the  content  of  each  of  the  courses  were 
generally  quite  favorable.  Of  the  total  of  104  topics  taught  across  the  four 
courses,  88  were  judged  to  be  moderately  or  more  useful  by  at  least  60  percent 
of  the  graduates.  Of  these,  29  were  rated  of  at  least  moderate  usefulness  by 
more  than  80  percent  of  the  respondents.  These  included  10  of  35  topics  from 
the  LMR  course,  particularly  the  sessions  dealing  with  grievances;  9 of  26 
topics  covered  in  the  CD  course,  especially  the  discussions  of  the  meaning 
and  purpose  of  discipline  (positive  versus  negative  discipline,  discipline 
and  motivation,  identifying  discipline  problems,  and  supervisory  roles);  and 
10  of  21  ME  topics,  most  notably  those  on  management  philosophy  and  style, 
group  processes,  management  communications  and  team  action. 

None  of  the  topics  in  the  PIP/PER  course  were  rated  remarkably  high  in 
terms  of  usefulness;  however,  there  were  also  no  particularly  low  ratings  for 
this  course  either.  The  topic  with  the  lowest  ratings,  the  relationship  of 
PIP/PER  to  other  FAA  programs,  was  judged  at  least  moderately  useful  by  less 
than  56  percent  of  the  respondents. 

With  the  exception  of  five  topics,  all  of  the  course  content  areas  were 
evaluated  as  useful  by  at  least  50  percent  of  the  graduates.  Four  of  these 
topics  were  in  the  LMR  course  and  referred  to  contract  negotiations,  a 
process  with  which  most  of  the  graduates  appear  to  have  very  little  contact. 
The  only  other  subject  rated  this  low  concerned  the  presentation  in  the  ME 
course  on  operation  of  assessment  centers. 

Self-reports  of  important  behavioral  changes  resulting  from  attendance  at 
these  courses  focused  on  four  main  areas  comnon  to  the  four  courses.  First, 
many  of  the  comments  spoke  to  an  increase  in  understanding  of  the  subject 
under  consideration  in  the  specific  course  and  an  increased  ability  to 
effectively  administer  agency  programs  in  these  areas.  There  were  also  many 
coninents  concerning  improvement  in  general  supervisory  and  managerial  skills, 
personal  growth  as  an  individual,  and  improved  human  relations. 

The  specific  behavioral  ratings  obtained  from  graduates,  their  immediate 
superiors,  and  supervisees  generally  confirmed  the  notion  that  the  courses 
had  beneficial  impacts  on  supervisory  behavior.  As  expected,  the  graduates 
themselves  were  most  likely  to  report  improved  behavior,  followed  fairly 
closely  by  the  reports  of  their  imnediate  superiors.  Among  graduates,  about 
two-thirds  to  three-fourths  of  the  respondents  on  the  average  reported 
increased  effectiveness  in  the  various  behaviors  of  concern  in  the  question- 
naires. The  range  for  inmediate  superiors  was  similar,  but  averaged  about  5 
percent  lower. 
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The  ratings  by  supervisees  were  less  laudatory,  but  on  the  average  about 
50  percent  of  the  respondents  reporting  increased  supervisory  effectiveness  in 
graduates . 

Diminished  effectiveness  following  MTS  training  was  rarely  reported  by  any 
of  the  three  groups. 

The  average  behavioral  ratings  for  three  of  the  four  courses,  PIP/PER,  CD, 
and  ME,  were  very  close  for  all  three  groups  of  respondents.  About  70  percent 
of  the  graduates,  65  percent  of  the  immediate  superiors,  and  47  percent  of  the 
supervisees  reported  improved  supervisory  performance.  The  corresponding 
values  for  the  LMR  course  were  significantly  higher  at  78,  75,  and  61  percent 
respectively. 

In  considering  the  specific  behavioral  effects  of  the  PIP/PER  course,  the 
three  groups  of  respondents  agreed  that  one  of  the  most  frequently  improved 
areas  of  functioning  concerned  awareness  of  the  effects  of  management  styles 
on  employees.  On  the  less  effective  side,  supervisees  tended  to  see  little 
improvement  in  actions  concerned  with  Major  Job  Assignments  (MJAs)  or  appeals 
of  performance  ratings. 

Graduates  of  the  LMR  course  tended  to  feel  that  not  only  was  the  course 
presentation  on  various  aspects  of  grievances  most  valuable,  it  was  also  the 
area  where  they  noticed  the  most  improvement  in  their  effectiveness  as  super- 
visors. The  immediate  superiors  of  graduates  felt  that  relatively  little  had 
been  accomplished  in  the  area  of  ability  to  deal  with  contract  negotiations. 
This  was  a teaching  area  rated  of  little  value  by  graduates. 

As  for  the  CD  course,  graduates  most  often  noted  change  in  the  area  of 
employing  informal  disciplinary  approaches.  The  most  frequent  change 
reported  by  supervisees  was  that  graduates  had  become  more  adept  at  employing 
approaches  to  encourage  self-discipline  and  were  more  aware  of  various 
symptoms  and  causes  of  discipline  problems. 

ME  graduates  most  frequently  felt  that  they  had  improved  in  the  areas  of 
communication  and  understanding  group  dynamics;  however,  supervisees  felt 
improvement  was  least  likely  to  be  noted  in  these  areas.  Graduates  and  their 
immediate  superiors  agreed  that  the  course  had  done  little  to  improve  the 
graduates'  ability  to  participate  effectively  in  the  assessment  center  process. 

The  impact  of  the  courses  on  conditions  within  the  unit  was  judged  by  both 
graduates  and  supervisees  to  be  primarily  in  the  areas  of  improved  relation- 
ships between  supervisors  and  employees  and  increased  unit  efficiency. 

One-half  to  three-fourths  of  the  respondents  were  inclined  to  report  such 
improvements.  Again,  graduates  were  more  likely  to  report  a positive  impact 
than  were  supervisees.  Considerably  fewer  respondents  in  either  group, 
usually  about  one-third, fe It  job  satisfaction  and  employee  morale  had  improved 
after  the  supervisor's  return  from  MTS  recurrent  training. 
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Analyses  were  conducted  on  all  items  concerning  differences  in  response 
trends  across  FAA  services  (AF,  AT,  FS , and  Others),  and  location  (head- 
quarters, regional  offices,  field  facilities).  There  were  occasional  differ- 
ences noted,  but  for  the  most  part  response  trends  were  consistent  across 
these  factors. 

Taken  together,  the  findings  suggest  that  the  MTS  courses  are  proving  to 
be  of  value  to  the  supervisors  and  managers  who  attend  them.  While  some 
adjustments  in  course  content  and  focus  are  suggested  by  this  evaluation,  the 
primary  conclusion  must  be  that  the  structure,  concept,  and  execution  of  the 
courses  is  generally  resulting  in  desired  outcomes,  at  least  as  viewed  by  the 
participants  in  this  survey. 


PART  TWO.  Survey  Results 


I.  Introduction. 

In  1973  an  assessment  of  the  effectiveness  of  the  basic  supervisory  and 
managerial  training  courses  provided  by  the  FAA  Management  Training  School 
(MTS)  was  undertaken  (3).  It  was  found  that  supervisors  and  managers  held 
similar  views  of  MTS.  Most  felt  that  the  3-week  course  at  MTS  had  been 
useful  in  helping  them  meet  the  demands  of  their  positions.  Ratings  by 
graduates  of  MTS,  their  immediate  superiors,  and  supervisees  indicated  that 
MTS  had  beneficial  effects  on  the  on-the-job  performance  of  the  graduates. 

Since  the  time  of  the  first  survey,  the  focus  of  training  at  MTS  has 
shifted.  While  the  basic  courses  continue  to  be  offered  to  new  supervisors 
and  managers,  most  of  the  training  is  now  directed  toward  upgrading  super- 
visory and  managerial  skills  through  recurrent  training.  It  is  the  recurrent 
training  program  that  is  the  concern  of  this  study. 

At  the  time  of  this  evaluation  ( June-September  1977),  the  core  of  the  MTS 
recurrent  training  program  consisted  of  four  courses,  each  of  which  was  1 
week  in  duration.  The  courses  are  designed  to  provide  a detailed  considera- 
tion of  one  of  the  topics  introduced  in  the  basic  courses.  They  are: 

Performance  Improvement  and  Employee  Appraisal  (PIP/PER)  - study  of 
agency  performance  improvement  and  appraisal  programs;  employee  counseling 
with  review  of  human  relations;  management  theory;  and  communication 
principles . 

Labor  Relations  for  Management  (LMR)  - study  of  union-management  rela- 
tions; contract  administration  and  negotiation;  unfair  labor  practices;  and 
grievances  and  arbitration. 

Constructive  Discipline  (CD)  - study  of  FAA  philosophy  and  policy  toward 
discipline;  motivation;  application  of  corrective  action;  and  disciplinary 
methods  available  for  use  by  supervisors. 
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Managerial  Effectiveness  (ME)  - study  of  management  theory,  group  dynamics, 
and  transactional  analysis  for  managers  with  emphasis  on  management-by- 
objectives and  team  action. 

The  evaluation  of  each  of  these  four  recurrent  training  courses  follows 
the  general  approach  taken  in  the  earlier  study  of  the  basic  MTS  courses. 

First,  the  effectiveness  of  the  content  of  each  course  was  evaluated  by 
graduates  of  the  respective  courses.  This  is  defined  as  an  "internal 
criterion"  measure  of  effectiveness  according  to  Martin  (2)  and  is  primarily  a 
measure  of  what  is  learned  in  the  courses.  The  evaluation  also  included 
measures  of  the  impact  of  the  courses  on  supervisory  or  managerial  behavior 
and  on  conditions  in  the  unit.  This  aspect  of  the  study  included  ratings  of 
changes  in  the  behavior  of  graduates  following  attendance  at  an  MTS  recurrent 
course.  These  ratings  were  made  by  the  graduates  themselves,  by  their 
immediate  superiors,  and  by  selected  supervisees  of  the  graduates.  Ratings  of 
unit  efficiency,  job  satisfaction,  and  morale  were  also  obtained.  These 
ratings  are  considered  "external  criteria"  in  that  they  focus  on  the  impact  of 
the  training  on  the  job  setting  rather  than  on  the  graduate.  Use  of  both 
approaches  is  basic  to  determining  the  effectiveness  of  supervisory  or 
managerial  training  according  to  Campbell,  Dunnette,  Lawler,  and  Weick  (1). 

II.  Method. 

A.  Questionnaires . Three  questionnaires  were  used  for  each  of  the  four 
recurrent  courses (Appendixes  I,  II,  and  III).  The  questionnaire  for  gradu- 
ates of  the  course  (Appendix  I)  consisted  of  five  parts:  (i)  demographic  data 
(e.g.,  age,  type  of  facility,  location),  (ii)  general  questions  about  the 
overall  value  of  the  course,  (iii)  a general  assessment  of  conditions  within 
the  supervisor's  organizational  unit,  (iv)  an  evaluation  of  the  specific 
content  of  the  course,  and  (v)  a self-assessment  of  the  impact  of  the  course 
on  supervisory  behavior  of  the  graduate.  The  ratings  of  course  content  were 
made  on  a five-point  scale  that  ranged  from  "not  at  all  useful"  to  "extremely 
useful."  The  behavior  rating  scales  also  contained  five  points  that  ranged 
from  "considerable  improvement"  to  "considerable  reduction"  in  effectiveness. 

The  other  two  questionnaires  were  designed  for  the  immediate  superior  of 
the  graduate  and  one  of  the  graduate's  supervisees  (Appendixes  II  and  III). 
These  questionnaires  included  demographic  and  behavioral  assessment  items  that 
paralleled  those  provided  course  graduates.  The  questionnaire  for  supervisees 
also  included  a section  for  evaluating  conditions  within  the  organizational 
unit . 

B.  Participants ■ Six  hundred  graduates  of  recurrent  training  (150  from 
each  course)  were  selected  to  participate  in  the  survey.  A stratified  random 
sampling  procedure  was  employed  that  controlled  the  distribution  of  partici- 
pants to  insure  proportional  representation  by  FAA  region  and  service  (Airway 
Facilities,  Air  Traffic,  Flight  Standards,  and  Others)  for  each  course.  Each 
of  the  selected  individuals  was  called  by  telephone  and  asked  to  participate 
in  the  study.  The  purpose  of  the  evaluation  was  explained  as  was  the  totally 
voluntary  nature  of  the  request.  Only  15  of  those  contacted  declined  to 


participate.  Another  174  had  moved,  had  retired,  or  were  otherwise  unavail- 
able. These  vacancies  were  filled  by  alternate  selections  until  150  graduates 
of  each  course  had  agreed  to  be  included  in  the  study. 

The  questionnaires  were  mailed  directly  to  each  individual  supervisor  who 
agreed  to  participate  in  the  evaluation.  (Note:  For  the  remainder  of  the 
report,  the  word  supervisor  should  be  understood  to  include  managers  unless 
managers  alone  are  being  discussed. ) The  questionnaires  for  immediate 
superiors  were  sent  along  with  the  forms  for  the  graduates  with  the  request 
that  they  be  forwarded  directly  to  their  immediate  superiors  by  the  graduates 
themselves.  Separate  return  envelopes  were  supplied  to  prevent  any  breach  in 
confidentiality. 

The  questionnaires  for  supervisees  were  distributed  in  a somewhat 
different  manner.  When  each  MTS  graduate  was  called  about  the  survey,  each 
was  asked  to  name  the  individuals  working  directly  under  his  or  her  super- 
vision. From  each  graduate's  list  of  supervisees  one  of  the  names  was 
selected  on  a random  basis  to  receive  a questionnaire.  The  questionnaire  was 
sent  directly  to  that  person  under  separate  cover.  This  procedure  was 
followed  to  insure  that  no  bias  would  be  involved  in  the  selection  of  super- 
visees to  participate  in  the  study. 

All  participants  were  asked  to  respond  as  quickly  as  possible  to  the 
survey  and  to  return  the  completed  forms  directly  to  the  principal  investi- 
gator in  the  envelopes  provided.  Approximately  two-thirds  of  the  sample  of 
graduates  contacted  for  each  of  the  courses  returned  their  questionnaires 
(Table  1).  These  return  rates  exceeded  those  required  for  95  percent  certainty 
that  the  estimates  provided  by  the  sample  were  within  plus  or  minus  10 
percent  of  the  values  that  would  have  been  obtained  from  surveying  all 
graduates . 


TABLE  1.  Ntmfcer  of  Questionnaires  Returned 


Course 

Respondent 

Craduate 

IoKdlate 

Superior 

Stq>ervlsee 

Total  Returned 

*33 

35* 

303 

Total  Usable* 

*02 

263 

280 

PIP/PER 

>8 

79 

6* 

LHR 

10* 

55 

70 

CO 

102 

69 

i A 

tc 

98 

60 

82 

*Some  questionnaires  returned  were  unusable  because  of 
insufficient  course  identification,  demographic,  or 
rating  data. 


6 


The  return  rates  for  immediate  superiors  and  supervisees  were  lower  than 
for  MTS  graduates  (Table  1).  This  was  expected  since  no  direct  contact  about 
participation  was  made  with  anyone  other  than  the  selected  graduate  super- 
visors. The  return  rates  for  the  four  courses  were  very  similar  for  gradu- 
ates but  differed  considerably  for  immediate  superiors  and  supervisees.  It  is 
not  clear  why  these  differences  in  returns  occurred  for  the  latter  two  groups 
since  there  was  no  relationship  between  questionnaire  length,  any  of  the 
demographic  variables,  or  other  factors  in  the  evaluation  and  the  numbers  of 
questionnaires  returned. 

The  age  and  experience  characteristics  of  the  participants  are  presented 
in  Table  2.  Generally,  the  values  for  each  group  are  comparable  to  the 
corresponding  values-  obtained  in  the  study  of  the  basic  MTS  courses  (3).  As 
with  the  basic  courses,  those  attending  the  ME  course  tend  to  be  somewhat 
older  than  those  attending  courses  designed  primarily  for  supervisors.  The 
group  of  immediate  superiors  was  somewhat  older,  except  for  those  in  the  CD 
course  sample.  For  each  course  the  supervisees  were  somewhat  younger  than 
the  graduates.  Since  these  are  the  expected  relationships,  it  simply  means 
that  the  samples  of  participants  were  not  unusually  distributed  on  these 
dimensions . 


TABLE  2.  Average  Age  and  Experience  In  Year*  of  Partlelpanta  In 
the  Evaluation  of  MTS  Recurrent  Courses 


Factors 

Course 

Ag« 

FAA  Experience 

Supervisory 

Experience 

PIP 

Graduates 

46.5 

17.4 

7.6 

Iaaedlate  Superiors 

48.3 

21.0 

12.5 

Supervisees 

42.2 

14.4 

LW 

Graduates 

45.8 

18.0 

8.5 

Iaaedlate  Superiors 

46.9 

21.3 

12.0 

Supervisees 

42.1 

13.8 

CO 

Graduates 

47.1 

18.6 

8.9 

Iaaediate  Superiors 

47.0 

23.5 

11.9 

Supervisees 

42.7 

12.0 

ME 

Graduates 

44.2 

21.2 

12.3 

Iaaediate  Superiors 

48.8 

22.7 

12.6 

Supervisees 

44.5 

12.7 

The  distribution  of  participants  across  various  demographic  (background) 
factors  is  shown  in  Table  3.  It  is  readily  apparent  that  the  graduates  were 
predominantly  male,  were  at  upper  grade  levels,  and  were  distributed  acrcos 
services  and  locations  in  general  proportion  to  the  actual  distribution  of  FAA 
personnel.  The  major  exception  to  this  was  the  relative  under-representation 
of  Flight  Standards  personnel  in  the  LMR  course.  Since  the  Flight  Standards 
Service  has  relatively  little  interaction  with  unions,  few  of  their  personnel 
attend  this  course. 
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TABLE  3.  Demographic  Distribution  of  Graduates  (C), 


sdlate  Stjperiors  (IS),  and  Supervisees  (S)  In  the 


Evaluation  of  NTS  Recurrent  Courses  (Totals  for  each  factor  may  vary  slightly 
due  to  Missing  Information  for  some  respondents.) 


Demographic 

Factor 

C 

pip 

is 

Se 

C 

or- 

is 

Se 

C 

TT" 

IS 

Se 

C 

IT 

IS 

Se 

Sex 

Male 

96 

78 

59 

101 

54 

61 

99 

68 

51 

98 

60 

71 

Female 

2 

1 

5 

3 

1 

9 

3 

1 

13 

0 

0 

11 

FAA  Service 

Alraay  Facilities 

33 

34 

27 

35 

18 

24 

36 

22 

23 

26 

15 

28 

Air  Traffic 

42 

27 

20 

59 

33 

34 

42 

29 

24 

35 

27 

20 

Fll^it  Standards 

11 

7 

4 

1 

0 

6 

16 

9 

9 

18 

9 

19 

Other 

12 

11 

13 

9 

4 

6 

9 

9 

10 

18 

9 

13 

Grade  Level 

1-7 

1 

0 

6 

! 

0 

5 

1 

0 

12 

0 

0 

16 

8-9 

2 

1 

4 

1 

0 

5 

0 

0 

3 

0 

1 

4 

10-11 

8 

1 

20 

12 

2 

24 

6 

1 

20 

0 

0 

6 

12 

29 

9 

21 

29 

6 

16 

28 

8 

20 

0 

1 

15 

13 

28 

13 

6 

20 

16 

13 

26 

9 

9 

9 

2 

23 

14 

21 

35 

5 

24 

15 

4 

26 

26 

0 

44 

20 

12 

15-16 

7 

20 

0 

13 

16 

1 

11 

23 

0 

43 

36 

2 

Location 

Headquarters 

3 

2 

2 

2 

0 

3 

4 

3 

0 

7 

3 

7 

Regional  Office 

10 

13 

6 

10 

9 

7 

10 

13 

7 

23 

23 

12 

Aeronautical  Center 

5 

5 

3 

5 

3 

4 

9 

7 

6 

5 

5 

7 

MAFEC 

1 

1 

1 

1 

1 

1 

3 

4 

5 

3 

0 

0 

Field 

79 

ia 

52 

86 

42 

55 

75 

41 

48 

60 

29 

52 

III.  Results. 

A.  PIP/PER  Course. 

1.  General  Evaluation  by  MTS  Graduates  (N  = 98). 

a.  Open-Ended  Questionnaire.  There  were  four  parts  to  the  open- 
ended  portion  of  the  questionnaire.  One  question  was  directed  at  identifying 
specific  changes  in  supervisory  behavior  and  is  reported  below  in  the  section 
concerning  behavior.  The  other  three  questions  concerned  the  effective  and 
ineffective  aspects  of  the  courses.  Answers  to  the  question  concerning  how 
the  course  could  have  been  more  helpful  and  the  question  on  how  it  could  be 
improved  were  largely  redundant.  Therefore,  only  the  recommendations  for 
improvement  are  discussed  below. 

The  most  helpful  aspect  of  the  course  according  to  the  respondents  (Table 
4)  was  the  acquisition  of  increased  knowledge,  particularly  in  the  areas  of 
understanding  the  Performance  Improvement  Program  (PIP)  and  understanding  of 
employee  needs,  concerns,  and  motivations.  Also  mentioned  frequently  were 
opportunities  for  personal  growth  (e.g.,  learning  more  about  self,  improving 
supervisory  practices,  increased  personal  effectiveness),  and  the  chance  to 
work  through  common  supervisory  problems  with  others. 
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TABLE  4.  Summary  of  Statements  Indicating  the  Host  Helpful  Aspects  of  the  PIP/PfR  Course 


Type  of  Statement 

Nusi>er 

of  Responses  by 

Service- 

Totals 

AF 

(N= 33) 

AT 

(N=42 ) 

FS 

(N=ll) 

0 

(N*12) 

Nueber  of 
Responses 

Percent 
of  Total 

Increased  Knowledge  and  Understanding 

S5 

59 

16 

15 

145 

66 

PIP  Program 

8 

15 

1 

- 

24 

11 

Understanding  fnployees/Muman  Relations 

7 

4 

6 

4 

21 

10 

Supervisory  Role 

9 

6 

1 

2 

18 

8 

Preparation  of  PER 

3 

7 

1 

- 

11 

5 

Review  of  S<4>ervisory  Principles 

6 

3 

- 

2 

11 

5 

Other 

22 

24 

7 

7 

60 

27 

Personal  Growth 

11 

23 

2 

3 

39 

18 

Working  Through  Supervisory  Problems  With  Others 

11 

14 

5 

2 

32 

15 

Miscellaneous 

1 

1 

- 

- 

■» 

1 

1 Af  - Airway  Facilities 
AT  - Air  Traffic 
FS  - Fli^it  Standards 
0 - Other 


There  was  considerable  divergence  of  opinion  about  the  kinds  of  improve- 
ments that  should  be  made  in  the  course  (Table  5).  The  greatest  number  of 
statements  concerned  some  aspect  of  course  content,  but  no  single  aspect  of 
content  was  identified  by  as  many  as  10  percent  of  the  respondents  as  needing 
improvement.  The  only  specific  areas  for  improvement  that  received  as  many  as 
10  percent  of  the  comments  concerned  a desire  to  reduce  the  mix  of  supervisors 
from  various  FAA  services  and  a desire  to  see  the  instructor  staff  improved, 
particularly,  with  respect  to  awareness  of  specific  FAA  programs  and  problems. 
Ten  respondents  had  no  recommendations  for  improvement  or  felt  the  course  was 
fine  in  its  present  state.  There  were,  however,  two  respondents  who  recom- 
mended eliminating  the  course  entirely. 


TABLE  5.  Simmary  of  Statement*  Indicating  Area*  of  Needed  Improvement  In  the  PIP/PER  Course 


Type  of  Statement 

Number  of  Responses  by  Service- 

Totals 

AF 

(N=33) 

AT 

(N*42) 

FS 

(Nxll) 

0 

(N=12) 

Number  of 
Responses 

Percent 
of  Totalb 

Course  Content 

14 

29 

2 

7 

5 2 

39 

Modify  Course  Approach 

6 

13 

3 

3 

25 

19 

More  practical  emphasis 

2 

2 

1 

5 

4 

Other 

4 

11 

3 

2 

20 

15 

Leave  Course  As  Is 

4 

2 

4 

0 

10 

8 

Improve  Instructors 

4 

2 

3 

1 

10 

8 

Miscellaneous 

13 

16 

i 

4 

36 

27 

*AF  - Airway  Facilities  bS<»  of  percentages  may  vary  slightly  from  total  due  to 

AT  - Air  Traffic  rounding  error. 

FS  - Fll^it  Standards 
0 - Other 


9 


-Ts*" 


4 


These  findings  suggest  that  the  course  as  a whole  has  no  particularly 
important  problem  areas  or  weaknesses  that  are  apparent  to  students. 

Instead,  there  are  a number  of  areas  that  are  noted  as  problems  for  some,  but 
not  others,  who  attended  the  course. 

b.  Usefulness  of  the  PIP/PER  Course.  The  overall  utility  of  the 
course  was  assessed  by  asking  each  respondent  to  rate  the  course  on  a five- 
point  scale  from  "very  useful"  to  "not  at  all  useful"  (See  Item  1 on  the 
questionnaires  shown  in  Appendix  I).  A total  of  91.8  percent  of  the  gradu- 
ates felt  the  course  was  moderately,  generally,  or  very  useful.  Only  one 
respondent  felt  the  course  was  of  no  value. 

2.  Evaluation  of  Course  Content. 

The  subjects  taught  in  this  and  the  other  three  courses  were 
evaluaced  in  terms  of  usefulness  to  the  graduate  supervisor  in  actual  work 
settings.  A five-point  scale  was  used  (Appendix  I)  that  ranged  from  "not  at 
all  useful"  through  "slightly,"  "moderately,"  and  "quite  useful"  to  "extremely 
useful." 

In  general,  extreme  ratings  were  not  common.  There  were  on  the  average 
between  five  and  six  ratings  at  the  low  ("not  at  all  useful")  and  again  at 
the  high  ("extremely  useful")  ends  of  the  scale  for  the  specific  instructional 
units  in  this  course.  The  average  proportion  of  respondents  who  felt  the 
course  topics  to  be  of  moderate  or  more  use  was  69.4  percent. 

There  were  also  only  three  topics  in  the  PIP/PER  course  for  which  the 
ratings  differed  between  FAA  services  or  the  locations  of  the  individuals.  These 
items,  one  each  in  the  instruction  blocks  on  Performance  Appraisal,  Management 
Theory , and  Performance  Improvement  Program,  are  discussed  below  in  the 
presentation  of  the  findings  concerning  topics  in  each  block  of  instruction. 

a.  Counseling.  The  instructional  block  on  Counseling  consisted 
of  only  the  one  topic  of  Principles  of  Counseling  which  was  rated  at  least 
moderately  useful  by  76  percent  of  the  respondents.  Of  the  22  specific 
subjects  taught  in  the  PIP/PER  course  (Table  6)  only  two,  Theory  and 
Principles  of  Communications  and  Approaches  to  Discussing  Performance 
Appraisals  (PER)  with  Employees,  were  more  highly  rated.  It  should  be  noted 
that  the  latter  topic  is  in  fact  the  application  of  the  counseling  process  to 
the  PER. 

b.  Communicat ions . The  ratings  for  the  specific  topics  covered 
by  this  instructional  block  were  uniformly  high  as  about  75  percent  of  the 
graduates  participating  in  the  study  felt  the  presentations  on  Communications 
Theory,  Communications  and  Management  style,  and  Application,  to  PIP/PER  were 
at  least  moderately  useful.  The  unit  on  Application  of  Communication  Prin- 
ciples to  Work  Settings  was  rated  moderately  or  more  useful  by  71  percent  of 
the  respondents. 
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TABLE  6.  Usefulness  Ratings  of  PIP/PER  Course  Content 


Content  Area 

Percent  rating 
Moderate,  quite, 
or  very  useful 

Counseling  Employees 

76.3 

Coaaaunication  Principles 

70.  5 

Theory  and  Principles  ol  CoaMunlcation 

77.7 

Application  to  PIP/PER  Review 

75.0 

CoaMunlcation  and  Managerial  Style 

74.2 

Application  to  Work  Setting 

71.3 

Perfonaance  Appraisal  (PER) 

69.7 

Discussing  Appraisals  With  Eag>loyees 

77.3 

Use  of  the  PER 

74.2 

Establishing  Perfonaance  Ratings 

74.2 

Purposes  of  Appraisal 

72.4 

Purposes  of  the  PER 

72.2 

Methods  of  Appraisal 

69.6 

Problems  In  Appraisal 

68.4 

Philosophy  of  Appraisal 

68.1 

Appeals 

65.3 

Relationship  to  Other  FAA  Prograan 

55.4 

Management  Theory 

65. 3 

Management  Style 

69.8 

Current  Theoretical  Approaches 

64.1 

Application  to  Perfonaance  Improvement 

62.1 

Perfonaance  Iaiproveaient  Prograai  (PIP) 

65.3 

Philosophy  of  PIP 

67.7 

Measuring  Results 

67.0 

Relation  to  Other  FAA  Programs 

64.5 

Development  of  Major  Job  Asslgnaients 

62.1 

c.  Performance  Appraisal.  The  ratings  for  the  10  topics  covered 
in  this  content  area  varied  considerably.  About  three-fourths  of  the  respond- 
ents felt  the  presentations  on  Approaches  to  Discussing  Appraisals  (77  percent), 
the  Use  of  the  PER  (74  percent),  the  Establishment  of  Performance  Ratings  (74 
percent),  and  the  Purposes  of  the  PER  (72  percent)  were  moderately  or  more 
useful.  On  the  other  hand,  only  somewhat  more  than  half  (55  percent)  of  the 
participating  graduates  felt  that  the  unit  on  the  Relationship  of  PER  to  Other 
FAA  Programs  was  of  moderate  or  better  use. 

The  topic  of  Methods  of  Appraisal  was  rated  more  useful  by  respondents  from 
Washington  Headquarters,  National  Aviation  Facilities  Experimental  Center,  and 
the  Aeronautical  Center  (HD  group)  than  by  those  from  Regional  Offices  (RO)  or 
field  facilities  (FF).  In  the  HD  group,  87.5  percent  felt  th . s subject  was 
moderately  or  more  useful,  while  70.0  percent  of  the  RO  and  t .0  percent  of  the 
FF  personnel  felt  the  same  way.  The  overall  chi-square  test!  was  significant 
at  the  .05  level,  as  were  the  differences  between  the  HD  group  and  both  the  RO 
and  FF  groups.  The  RO  and  FF  groups  did  not  differ.  This  suggests  that  the 
closer  the  supervisor  is  to  the  field,  the  less  relevant  this  specific  topic 
is  to  his  or  her  job. 


^-The  chi-square  statistic  was  used  throughout  these  analyses  to  determine 
whether  or  not  the  proportions  of  respondents  giving  favorable  ratings  on  each 
survey  item  were  equal  for  all  groups. 
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d.  Management  Theory.  The  ratings  of  the  presentations  in  this 
area  differed  relatively  little.  About  two-thirds  of  the  respondents  judged 
each  of  the  three  units  to  be  of  moderate  or  better  use. 

One  of  the  topics  in  this  block,  the  presentation  on  Current  Theoretical 
Approaches  to  Supervision,  was  rated  somewhat  differently  as  a function  of 
the  location  of  the  respondent.  All  of  the  HD  graduates  (100  percent)  rated 
this  topic  to  be  at  least  moderately  useful.  The  corresponding  values  for 
the  RO  and  FF  groups  were  70.0  percent  and  58.9  percent  respectively.  An 
overall  significant  chi-square  test  (£  < .05)  resulted  from  the  significant 
difference  between  the  HD  and  FF  groups  (£  < .05).  The  HD-RO  difference 
approached  but  did  not  achieve  statistical  significance.  There  was  no 
difference  between  the  RO  and  FF  groups.  Again,  as  noted  for  the  item  on 
methods  of  appraisal,  the  supervisors  in  the  field  appeared  to  feel  this 
topic  to  be  less  relevant  to  their  needs  than  did  HD  supervisors. 

e.  Performance  Improvement  Program.  As  with  the  items  on 
Management  Theory,  the  four  topics  under  the  PIP  area  were  rated  rather 
uniformly.  Approximately  two-thirds  of  the  respondents  rated  the  units  as  at 
least  moderately  useful. 

One  item,  Development  of  Major  Job  Assignments  (MJAs),  was  rated 
differentially  as  a function  of  both  FAA  service  and  respondent  location. 

With  respect  to  FAA  service,  75.8  percent  of  the  Airway  Facility  (AF) 
graduates  and  65.0  percent  of  the  combined  Flight  Standards  and  other  types 
of  graduates  (FS/0)  rated  this  topic  as  at  least  moderately  useful.  Only 
50.1  percent  of  the  Air  Traffic  (AT)  participants  gave  the  same  ratings.  The 
overall  significance  of  the  chi-square  test  (£  < .05)  was  the  result  of  the 
AT  proportion  being  significantly  (£  < .05)  lower  than  either  the  AF  or  FS/0 
proportions.  The  AF  and  FS/0  groups  did  not  differ.  The  differences  here 
probably  reflect  the  perception  of  AT  supervisors  that  they  operate  with 
standardized  MJAs  that  they  have  little  opportunity  to  develop  or  modify. 

As  for  location,  87.5  percent  of  those  from  the  HD  group  rated  this  topic 
as  moderately  or  more  useful.  For  the  RO  group  the  proportion  was  66.7 
percent  and  for  FF  supervisors  it  was  59.0  percent.  The  HD  group  had  a 
significantly  higher  percentage  according  to  a chi-square  analysis  of  such 
ratings  (£  < .05)  than  the  other  two  groups  which  did  not  differ  from  each 
other.  Considering  the  higher  degree  of  involvement  in  setting  MJAs  of  those 
in  the  HD  and  RO  locations  than  of  those  at  the  field  level,  the  findings 
should  not  be  considered  surprising. 

3.  The  PIP/PER  Course  and  Supervisory  Behavior, 
a.  Self-Assessment  (N  = 98). 

(1)  Most  important  changes.  The  graduates  were  asked  to 
indicate  how  their  own  supervisory  behavior  had  changed  after  attending  the 


12 


PIP/PER  course.  The  most  frequent  type  of  statement  (35  percent)  concerned 
some  aspect  of  generally  improved  supervisory  skills  (Table  7);  for  example, 
better  communications  with  employees  or  improved  ability  to  motivate  employees. 
The  second  most  frequent  type  of  response  (26  percent)  concerned  the  acquisi- 
tion of  some  specific  knowledge  applicable  to  the  supervisory  task.  The  bulk 
of  these  comments  (31  of  42)  concerned  increased  awareness  of  the  proper 
manner  in  which  to  proceed  with  the  PIP/PER  process.  Several  (19)  of  the 
participating  graduates  felt  that  they  had  benefited  in  a personal  way  from 
the  course  through  acquisition  of  greater  empathy,  self-confidence,  and  self- 
understanding. On  the  other  hand,  about  the  same  number  (18)  reported  that 
the  course  had  had  no  effect  on  their  supervisory  behavior.  The  remaining 
comments  concerned  improved  understanding  of  human  relations  and  employee 
motivations  (9  percent)  an'd  a better,  more  balanced  understanding  of  the 
supervisory /managerial  role  (6  percent). 


TABLE  7.  Stanary  of  Stataaents  Concerning  Specific  Changes  in  Supervisory  Behavior  Resulting  Fro*  Attendance  at  PIP /PEA  Course 


Nu*er  of  Responses  by  Service*  Totals 


Type  of  Stataaent 

* 

(H-33) 

AT 

(H.%2) 

FS 

(M.11) 

0 

(H*12) 

Nuder  of 
Responses 

Percent 
of  Total 

lap roved  general  supervisor*  skills 

21 

28 

3 

2 

58 

35 

Increased  knowledge  of  specific  application 

of  PIP /PER  program  to  s*g»ervisory  situations 

17 

22 

2 

1 

82 

28 

Conduct  of  PIP/PER 

12 

18 

0 

1 

31 

20 

Other 

5 

A 

2 

0 

11 

8 

Personal  growth 

8 

8 

2 

1 

19 

12 

Ho  changes  reported 

2 

7 

8 

3 

18 

12 

Increased  understanding  of  and  i^> roved 

hiaan  relations 

3 

8 

1 

2 

1* 

7 

Better  understanding  of  sipervisory  role 

8 

1 

- 

1 

10 

8 

A F - Airway  Facilities 
AT  - Air  Traffic 
FS  - Flight  Standards 
0 - Other 


(2)  Behavioral  ratings.  There  were  37  specific  behavioral 
outcomes  specified  for  the  PIP/PER  course  (Table  8).  Across  those  items  an 
average  of  69.5  percent  of  the  respondents  reported  improvement  in  effective- 
ness as  a result  of  attending  this  course.  The  number  reporting  reduced 
effectiveness  in  functioning  in  any  behavioral  area  was  never  more  than  five 
and  averaged  less  than  three  (2.9)  persons. 


In  looking  at  the  specific  behavioral  items,  the  two  with  the  greatest 
number  of  respondents  reporting  improvement  were  the  items  concerning  under- 
standing of  one's  owr.  supervisory/managerial  style  (#1)  and  understanding  the 
effects  of  various  supervisory  styles  on  employees  (#2).  The  proportion  of 
respondents  reporting  improvement  on  these  two  items  was  86  percent  for  each 
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item.  There  were  six  other  behavioral  items  for  which  at  least  75  percent  of 
the  respondents  indicated  that  improved  performance  occurred  as  a result  of 
the  course.  These  were  the  areas  of  applying  principles  of  communications  to 
relations  with  employees  (#6),  the  presentation  of  information  to  employees 
in  a helpful  manner  (#8),  attending  to  employee  needs  in  supervisory  actions 
(#15),  understanding  of  the  PER  (#29),  the  ability  to  properly  prepare  the  PER 
(#30),  and  understanding  of  requirements  for  denial  of  within-grade  increments 
(#34). 

There  were  only  two  items  for  which  less  than  60  percent  of  the  respond- 
ents reported  improvement.  On  the  item  concerning  their  understanding  of 

their  own  motivations  as  a supervisor  (#4),  60  percent  of  the  respondents  felt  < 

that  improvement  had  occurred.  Only  50  percent  felt  that  the  course  had 
helped  them  improve  their  understanding  of  the  PIP  in  relation  to  other  FAA 
programs  (#10). 

There  were  only  two  items  on  which  the  ratings  differed  across  FAA 
services.  A greater  percentage  of  AT  (78.6  percent)  and  AF  (66.7  percent) 
graduates  reported  improvement  on  the  item  concerning  the  definition  of  the 
objectives  of  counseling  sessions  (#22)  than  did  those  in  FS  and  other 
services  (47.8  percent).  A similar  pattern  was  noted  for  ratings  of  the 
1 adequacy  of  evaluation  of  employee  performance  (#36),  as  71.4  percent  of  the 

AT  and  66.7  of  the  AF  respondents  reported  improvement,  while  59.1  percent  of 
the  FS  and  other  graduates  felt  the  same  way.  The  differences  in  percentages 
between  the  AT  and  FS/0  graduates  were  significant  at  the  .05  level  according 
to  chi-square  analyses.  The  AF-FS/0  differences  approached  but  did  not 
achieve  statistical  significance.  The  AT-AF  comparisons  were  not  significant 
for  either  item.  The  differences  observed  for  both  of  these  items  may 
reflect  a greater  emphasis  on  the  tasks  of  evaluating  and  counseling 
employees  in  the  AT  and  AF  services,  possibly  because  of  the  presence  of 
unions. 

There  were  also  two  items  for  which  the  percentage  of  reported  behavior 
change  varied  as  a function  of  location  of  the  respondent.  The  HD  group 
(89.0  percent)  and  the  FF  group  (75.0  percent)  were  much  more  likely  than  the 
RO  group  (40.0  percent)  to  report  improvement  in  their  understanding  of 
* employee  motivation  (#5);  the  differences  between  RO  and  the  other  two  groups 

of  graduates  were  significant  at  the  .05  level  according  to  chi-square  tests. 

The  same  pattern  was  observed  on  the  item  concerning  application  of 
appropriate  principles  and  techniques  to  employee  counseling  (#20).  In  this 
case  78.0  percent  of  the  HD  group  and  73.0  percent  of  the  FF  group  reported 
r improvement,  while  only  30.0  percent  of  the  RO  graduates  made  such  reports. 

Again,  the  HD-RO  and  FF-RO  comparisons  were  significant  (£  < .05).  It  should 
k be  noted  that  the  number  of  respondents  in  the  HD  and  RO  groups  was  not  large 

and  thus  these  findings,  which  have  no  clear  explanation,  should  be  viewed 
conservatively. 
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b.  Assessment  by  Immediate  Superiors  (N  = 79).  On  the  whole, 
immediate  superiors  noted  less  change  in  the  behavior  of  graduates  (Table  8) 
than  did  the  graduates  themselves  (£  < .05)2.  The  average  proportion  of  the 
immediate  superiors  indicating  change  on  any  one  item  was  64.6  percent. 

The  only  performance  improvement  in  graduates  reported  by  more  than  75 
percent  of  the  immediate  superiors  was  understanding  the  effects  of  various 
managerial  styles  on  employees  (#2).  There  were  six  additional  areas  in 
which  at  least  70  percent  of  the  immediate  superiors  noted  improvement  in  the 
graduate  supervisors.  These  included  relating  managerial  theory  to  applied 
situations  (#3),  understanding  the  relationship  of  performance  improvement  and 
major  job  assignments  (#11),  understanding  the  relationship  of  PER  to 
performance  improvement  (#18),  understanding  the  relationship  of  appraisal  to 
other  FAA  programs  (#28),  understanding  of  the  PER"(#29),  and  evaluating 
employee  performance  fairly  (#37). 

The  areas  where  least  improvement  was  noted  by  immediate  superiors  were 
presenting  information  to  employees  in  a clear  fashion  (#7),  accurate  identi- 
fication of  MJAs  (#12),  measurement  of  results  in  MJAs  (#14),  developing 
well-defined  performance  standards  (#26),  and  soliciting  employee  involvement 
in  development  of  employee  standards  (#35).  However,  only  on  the  latter  item 
did  the  percentage  of  immediate  superiors  rating  at  least  some  improvement  in 
graduates  fall  below  50  percent. 

Only  rarely  did  immediate  superiors  rate  the  performance  of  graduates  in 
these  areas  as  diminished.  This  occurred  only  eight  times  for  the  36 
behavioral  items  and  never  more  than  once  an  item.  In  addition,  these  eight 
ratings  involved  only  three  graduates  of  the  course. 

There  were  no  differences  in  ratings  related  to  either  FAA  service  or 
location  of  employment . 

c.  Assessment  by  Supervisees  (N  = 64).  Of  the  three  groups 
rating  the  behavioral  effects  of  this  course  on  graduates,  the  ratings  of 
supervisees  reflected  significantly  less  perceived  change  than  those  by  either 
the  graduates  (£  < .01)2  or  superiors  (£  < .01)2.  On  the  average,  somewhat 
less  than  half  (46.1  percent)  of  the  supervisees  reported  observing  improve- 
ments in  the  functioning  of  graduates  in  the  PIP  and  PER  areas. 

Diminished  effectiveness  was  rarely  reported  and  averaged  only  slightly 
more  than  one  such  rating  (1.2)  per  item,  with  two  items  (development  of 
performance  standards  (#26)  and  relating  appraisal  to  other  FAA  programs 
(#28))  having  three  such  ratings  each. 


2 As  determined  by  Student's  (£)  test  of  the  differences  between  two  means. 
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There  were  10  behavioral  areas  (Table  8)  that  were  rated  as  showing 
improvement  by  at  least  50  percent  of  the  raters.  Of  these,  supervisory  style 
(#2),  proper  preparation  of  the  PER  (#30),  and  effective  conduct  of  the 
performance  review  (#31)  were  most  frequently  judged  by  supervisees  to  have 
shown  improvement  after  the  graduate  attended  the  PIP/PER  course. 

At  the  other  extreme,  six  areas  were  rated  improved  by  less  than  40 
percent  of  the  supervisees.  These  included  indication  of  results  expected  in 
MJAs  (#13),  measurement  of  results  obtained  in  MJAs  (#14),  clearly  stating 
the  purposes  of  appraisal  (#23),  and  development  of  well-defined  performance 
standards  (#26).  Considerably  below  these  were  judgments  about  the  presenta- 
tion of  the  appeals  process  (#32)  and  role  taking  in  the  appeals  process  (#33). 
Only  23  percent  of  the  graduates  were  judged  to  have  improved  on  the  former 
and  29  percent  on  the  latter. 

There  were  no  differences  betveen  the  ratings  of  supervisees  as  a function 
of  either  FAA  service  or  location. 

d.  Comparing  the  Three  Sets  of  Ratings.  As  noted  above,  gradu- 
ates were  the  most  likely  of  the  three  groups  to  report  improvement  in  super- 
visory effectiveness  (about  70  percent).  The  immediate  superiors  were 
significantly  less  likely  to  do  so,  but  the  proportion  reporting  change  was 
not  dramatically  different  (about  65  percent)  from  that  of  the  graduates. 
However,  supervisees  were  considerably  less  likely  to  report  improvement  than 
either  of  these  two  groups  (about  46  percent). 

Ranking  the  items  from  most  to  least  reported  behavior  change  and  corre- 
lating these  ranks  for  the  responding  groups  showed  that  graduates  and 
immediate  superiors  showed  negligible  agreement  as  to  which  areas  showed  most 
improvement  (rs  = .25,  not  signi f icant 3) . The  comparison  between  graduates 
and  supervisees,  on  the  other  hand,  revealed  a considerable  similarity  in 
ranking  (rs  = .54,  £ < .01).  In  other  words,  graduates  and  supervisees  tended 
to  see  the  relatively  greater  or  lesser  change  in  supervisory  performance  in 
the  same  areas. 

Looking  at  the  specific  areas  of  agreement,  all  three  groups  more 
frequently  reported  improvement  in  understanding  the  effects  of  supervisory 
styles  on  employees  than  in  most  other  behavioral  areas.  The  three  groups  did 
not  tend  to  agree  as  to  the  other  areas  of  performance  that  had  shown  the  most 
relative  improvement. 

Graduates  and  their  immediate  superiors  showed  agreement  on  the  relative 
frequency  with  which  they  reported  improved  ability  to  relate  managerial  theory 
to  the  actual  supervisory  situation.  Graduates  and  supervisees  noted  rela- 
tively frequent  improvement  in  the  effectiveness  of  the  PER  review  with 
supervisees . 


^All  correlations  reported  were  calculated  by  the  Spearman  method  of 
rank-order  correlation. 
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In  terms  of  least  reported  change,  both  immediate  superiors  and  super- 
visees felt  that  relatively  little  change  had  occurred  in  accuracy  of  identi- 
fication of  Major  Job  Assignments  (MJAs)  or  in  presenting  information  to 
employees  in  a clear  fashion.  This  latter  area  was  among  those  for  which 
graduates  were  most  likely  to  report  improvement.  This  suggests  a notable 
discrepancy  in  perceptions  about  the  effectiveness  with  which  this  task  is 
accomplished. 

4.  Impact  on  the  Unit. 

Both  graduates  and  their  supervisees  were  asked  to  indicate  to 
what  extent  conditions  within  their  units  had  changed  as  a function  of  the 
graduates'  attendance  at  the  PIP/PER  course.  In  general  the  assessments  of 
the  two  groups  were  remarkably  similar  (Table  9).  Over  three-fourths  of  the 
graduates  and  about  two-thirds  of  the  supervisees  felt  the  supervisor's  inter- 
actions with  employees  had  improved,  and  a majority  of  both  groups  felt 
efficiency  within  the  unit  had  increased.  However,  ratings  of  job  satisfac- 
tion and  unit  morale  suggested  less  change  on  these  dimensions  as  only  30  to 
40  percent  of  the  respondents  in  both  groups  felt  improvements  had  occurred 
in  these  areas. 


TABLE  9.  Percentage  of  Graduates  and  Supervisees  Rating  Conditions  In 


Their  Unit  as  Improved  After  Attendance  of  Graduate  at  PIP/PER  Course 


I to* 

Graduate 

(N=98) 

Sipervlsee 

(N=64) 

Unit  Efficiency 

58.2 

51.6 

Employee  Oob  Satisfaction 

42.8 

35.9 

Employee  Morale 

31.6 

29.’ 

Supervisor's  Interaction  With  Employees 

79.6 

65.6 

Ratings  indicating  deterioration  of  conditions  within  units  were  rare.  No 
such  ratings  were  given  by  the  graduate  group  on  the  item  concerning  efficiency 
or  by  the  supervisee  group  on  the  efficiency  and  job  satisfaction  items. 

There  were  three  graduates  and  five  supervisees  who  felt  morale  had  decreased, 
and  three  supervisees  who  felt  the  "equal  ity  of  the  interaction  between 
supervisor  and  employees  had  declined. 
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B.  LMR  Course. 


1.  General  Evaluation  by  MTS  Graduates  (N  = 104). 

a.  Open-Ended  Questionnaire.  The  most  helpful  feature  of  the  LMR 
course  (Table  10)  was the increase in  general  knowledge  and  understanding  of 
LMR  gained  by  the  graduates.  The  specific  aspect  of  the  course  most 
frequently  cited  as  helpful  was  the  training  on  management  of  unfair  labor 
practices  (ULPs),  grievances,  and  appeals,  particularly  for  AT  supervisors. 

As  with  the  PIP/PER  course,  personal  growth  and  the  opportunity  to  work 
through  supervisory  problems  with  others  were  also  frequently  cited  as  being 
most  helpful. 


TABLE  10.  Suaaary  of  Statements  Indicating  the  Host  Helpful  Aspects  of  the  LMR  Course 


0 

3> 

e 

1 

w 

• 

• 

Service* 

Totals 

Type  of  Statement 

AF 

AT 

FS 

0 

Number  of 

Percent 

(M.35) 

(N*57 ) 

(Msl ) 

(N=9) 

Responses 

of  Total 

Increased  Knowledge  and  Understanding 

59 

2 

13 

176 

79 

LMR  In  General 

32 

V * 

1 

7 

76 

34 

ULPs,  Grievances  and  Appeals 

8 

l 22 

1 

2 

33 

15 

General  Principles  of  Supervisors 

7 

n 

0 

0 

24 

11 

Understanding  Employees/Huean  Relations 

7 

i n 

0 

2 

20 

9 

Union  Contracts 

2 

10 

0 

1 

13 

6 

Supervisory  Role 

j 

6 

0 

1 

10 

4 

Working  Through  Supervisory  Problems  With  Others 

6 

15 

0 

0 

21 

9 

Personal  Growth 

12 

8 

0 

0 

20 

9 

Miscellaneous 

2 

A 

0 

1 

7 

3 

AT  - Airway  Facilities 
AT  - Air  Traffic 
FS  - Flight  Standards 
0 - Other 


The  items  mentioned  as  needing  improvement  covered  a variety  of  areas 
(Table  11).  The  only  particular  item  mentioned  by  as  many  as  20  percent  of 
the  respondents  was  the  desire  for  more  discussion  on  the  specific  contract 
governing  the  work  situation.  This  was  particularly  true  of  AT  supervisors. 
The  next  most  frequent  response  was  that  the  course  should  not  be  changed. 
There  were  also  several  comments  requesting  more  role  playing  and  a more 
practical  emphasis  in  the  course,  otherwise  the  suggestions  ranged  over  many 
topics  with  no  clear  consensus  being  evident. 

Taken  together,  these  responses  indicate  that  focused  study  on  LMR  is  seen 
as  valuable  and  the  only  substantive  change  that  would  be  likely  to  increase 
that  value  is  in  the  area  of  giving  more  attjtoLion  to  the  specific  FAA-union 
contracts.  .; 
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TABLE  11.  Suemary  of  Statements  Indicating  Areas  of  Needed  Improvement  In  the  LMR  Course 


Type  of  Statement 

Number  of  Responses  by 

Service3 

Totals 

AT 

(N=35) 

AT 

(N=57) 

FS 

(N=l ) 

0 

(N=9 ) 

Number  of 
Responses 

Percent 
of  Total® 

Course  Content 

23 

50 

0 

5 

78 

51 

More  on  Specific  FAA-Union  Contracts 

6 

19 

0 

0 

25 

16 

More  on  Role-Playing  LMR  Situations 

5 

8 

0 

0 

13 

Other 

12 

23 

0 

5 

40 

26 

Modify  Course  Approach 

8 

23 

0 

1 

32 

21 

More  Practical  Emphasis 

2 

9 

0 

0 

11 

Other 

6 

14 

0 

1 

21 

14 

Leave  Course  As  Is 

7 

8 

1 

1 

17 

11 

Miscellaneous 

12 

15 

0 

0 

27 

18 

*AF  - Airway  Facilities  bSum  of  percentages  may  vary  slightly  from  total  due  to  rounding  error. 

AT  - Air  Traffic 
FS  - Flight  Standards 
0 - Other 

b.  Usefulness  of  the  LMR  Course.  On  the  overall  scale,  the 
course  was  rated  moderately  to  very  useful  by  90.4  percent  of  the  graduates. 
All  of  the  respondents  felt  the  course  was  of  some  value. 

2.  Evaluation  of  Course  Content. 

On  the  average,  68.4  percent  of  the  respondents  felt  the  topics 

covered  by  the  LMR  course  to  be  of  moderate  or  better  use.  This  is  a value 

similar  to  that  obtained  for  the  PIP/PER  course.  However,  in  contrast  to  the 
PIP/PER  course,  the  graduates  were  much  more  likely  to  provide  extreme 
ratings  on  the  subject  matter  in  the  LMR  course.  An  average  of  nearly  nine 
respondents  per  item  gave  ratings  of  "extremely  useful,"  and  for  one  item 
(discussed  below),  21  of  the  ratings  were  at  this  level.  On  the  "not  at  all 

useful"  end  of  the  scale,  the  number  of  such  ratings  averaged  11  per  item. 

Furthermore,  three  items  were  so  rated  by  more  than  30,  and  another  two  items 

were  so  rated  by  more  than  20  supervisors  (see  below).  In  other  words,  if 

the  graduates  found  a part  of  the  course  useful,  there  was  a good  probability 

that  it  would  be  seen  as  very  useful;  if  not,  it  was  quite  likely  to  be  rated 

as  having  no  value  at  all. 

There  were  only  three  topics  in  the  LMR  course,  one  each  in  the  instruc- 
tional blocks  concerning  Grievance , LMR  and  Communications,  and  Arbitrat ion , 
that  were  rated  differently  by  graduates  in  the  various  FAA  services  or  in 
the  different  locations  considered  in  the  survey.  Those  differences  are 
discussed  below  with  the  other  findings  for  the  topics  in  each  block  of 
instruction. 


20 


TABLE  12.  Usefulness  Retings  of  LHR  Course  Content 

Percent  rating 


Content  Area  Moderate,  quite, 

or  very  useful 


Grievances  82.7 

Documenting  Grievances  89.3 

FAA  Grievance  System  86.3 

Processing  Grievances  84.5 

Nature  of  Grievances  83.7 

Union  Involvement  in  Grievances  81.7 

CSC  Guidelines  77.7 

Negotiated  Grievance  System  75.7 

Nature  of  Uni on /Management  Relations  81.5 

Role  of  Supervisor  in  LHR  86.5 

Types  of  LHR  Relations  76.5 

Dealing  With  Unions  Under  Contract  79.7 

Roles  of  Management  81.6 

Roles  of  Unions  81.6 

Written  Agreements  vs.  Unwritten  Understandings  76.0 

Contract  Adel nls tret ion  76.0 

Actalnl strati on  of  Contracts  76.5 

Understanding  Contracts  75.5 

Llf*  Background  and  Overview  75.5 

FAA  Policy  and  Philosophy  of  LMR  81.7 

Executive  Order  11491  69.2 

Unfair  Labor  Practices  (ULPs)  73.1 

Understanding  ULP  Actions  75.0 

Procedures  for  ULPs  71.2 

LMR  and  Communications  70.7 

Employee  Rights  89.4 

Relations  Under  Exclusive  Union  Recognition  78.8 

Relations  Under  National  Consultation  Rights  63.0 

Relations  With  No  Union  Recognition  51.5 

Arbitration  57.9 

Alternatives  to  Arbitration  61.8 

Supervisory  Roles  in  Arbitration  60.2 

Uses  of  Arbitration  58.2 

Mechanics  of  Arbitration  57.3 

Preparing  Grievance  Cases  for  Arbitration  55.3 

Arbitration  Hearings  54.4 

Operations  in  a Union  Facility  57.2 

Dealing  With  Unions  73.5 

ULPs  70.6 

Contract  Adwlnlstratlon  63.4 

Negotiating  Contracts  21.4 

Contract  Negotiations  39.6 

Post-Negotiation  Responsibilities  47.4 

The  Negotiation  Process  36.6 

Action  Prior  to  Negotiation  34.6 


a.  Grievances . The  topics  taught  in  this  block  (Table  12)  were 
well  regarded  by  graduates  as  75  percent  or  more  of  the  ratings  for  each  topic 
were  at  the  moderately  or  more  useful  level.  In  fact,  five  of  the  seven 
topics  were  rated  as  at  least  moderately  useful  by  more  than  80  percent  of  the 
supervisors.  On  one  item,  Documentation  of  Grievances,  21  graduates  (20 
percent)  rated  the  topic  as  "extremely  useful."  This  was  the  greatest  propor- 
tion of  such  ratings  for  any  of  the  topics  taught  in  these  four  recurrent 
courses . 

The  discussion  of  Negotiated  Systems  was  rated  somewhat  differently  by  AF 
supervisors  than  by  the  graduates  from  other  services.  A total  of  69  percent 
of  the  AF  participants  felt  this  subject  was  at  least  moderately  useful, 
while  about  79  percent  of  the  AT,  FS , and  other  graduates  felt  the  same  way. 
The  differences  between  groups,  while  statistically  significant  according  to 
chi-square  analysis  (£  < .05)  were  not  so  large  as  to  suggest  dramatic 
differences  in  the  perceived  value  of  the  topic. 

b.  Nature  of  Union/Management  Relations.  The  discussion  of  the 
Role  of  Supervisors  in  LMR  was  rated  moderately  or  more  useful  by  87  percent, 
and  the  topic  of  LMR  Relationships  by  76  percent  of  the  respondents. 

c.  Dealing  With  Union  Under  Contract.  In  this 

unit,  each  of  the  three  topics  was  judged  of  considerable  utility.  The  items 
concerning  Roles  of  Management  and  Roles  of  Unions  were  each  rated  as  at 
least  moderately  useful  by  82  percent  of  the  graduates.  The  area  of  Written 
Agreements  versus  Unwritten  Understandings  was  judged  moderately  or  more 
useful  by  76  percent  of  the  respondents. 

d.  Contract  Administration.  The  two  topics  in  this  block  were 
considered  at  least  moderately  useful  by  about  three-fourths  of  the  graduates. 

e.  LMR  Background  and  Overview.  There  was  some  variation  in  the 
perceived  value  of  the  two  specific  topics  covered  in  this  unit.  Over  80 
percent  of  the  graduates  felt  the  presentation  of  FAA  Policy  and  Philosophy  of 
LMR  (82  percent)  was  at  least  moderately  useful.  A total  of  69  percent  felt 
the  discussion  of  the  Executive  Order  on  LMR  had  been  moderately  or  more 
useful . 


f.  Unfair  Labor  Practices  (ULPs).  The  two  discussions  designed 
to  increase  understanding  of  ULPs  were  judged  of  essentially  equal  utility. 
Three-fourths  of  the  respondents  felt  the  subject  of  Understanding  ULPs  was  at 
least  moderately  useful,  while  71  percent  felt  the  same  about  the  presentation 
on  Procedures  for  ULPs. 

g.  LMR  and  Communications.  There  was  considerable  diversity  in 
the  judged  value  of  the  four  topics  presented  in  this  unit.  The  discussion 
of  Employee  Rights  was  judged  moderately  or  more  useful  by  nearly  90  percent 
of  the  graduates  (89  percent).  In  contrast,  just  over  half  (51  percent)  felt 
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the  presentation  on  LMR  with  no  union  recognition  was  involved  was  helpful. 
More  than  three-fourths  (79  percent)  of  the  respondents  felt  the  discussion 
of  relations  with  exclusive  union  recognition  was  at  least  moderately  useful, 
and  about  two-thirds  (63  percent)  felt  the  same  way  about  the  item  concerning 
relations  with  unions  when  national  consultation  rights  have  been  accorded. 

The  lowest  rated  item,  Labor  Relations  Without  Union  Recognition,  showed 
considerable  diversity  in  ratings.  The  most  frequent  response  on  this  item 
was  "quite  useful"  (29  percent),  but  there  were  also  many  who  felt  that  the 
topic  was  "not  at  all  useful"  (22  percent).  This  diversity  was  due 
primarily  to  the  fact  that  only  33.9  percent  of  the  AT  group  felt  this  topic 
to  be  of  moderate  or  more  use  compared  to  77.1  percent  of  the  AF  group  and 
60.0  percent  of  the  FS/O  groups.  The  differences  between  the  AT  and  other 
groups  were  highly  significant  by  chi-square  tests  (j>  < .001).  In  fact  21  of 
the  23  persons  rating  the  topic  as  having  no  usefulness  were  from  the  AT 
service . 

h.  Arbitration.  The  ratings  of  the  topics  in  this  content  area 
were  relatively  uniform,  ranging  from  54  percent  (Arbitration  Hearing)  to  62 
percent  (Alternatives  to  Arbitration)  of  the  graduates  judging  the  subjects 
to  have  moderate  or  better  usefulness. 

The  item  concerning  the  Arbitration  Hearing  received  somewhat  different 
ratings  from  FF  supervisors  than  from  those  in  RO  and  HD  locations  according 
to  chi-square  analysis  (£  < .05).  A total  of  60  percent  of  the  FF  respond- 
ents judged  this  topic  to  be  at  least  moderately  useful  compared  to  30 
percent  of  the  RO  and  25  percent  of  the  HD  groups. 


i.  Operations  in  a Union  Facility.  Two  of  these  presentat ions , 
Dealing  With  Unions  and  Unfair  Labor  Practices,  were  seen  as  moderately  or 
more  useful  by  at  least  70  percent  of  the  graduates.  The  area  of  Contract 
Administration  was  considered  less  useful  overall  but  still  was  given 
moderate  or  better  ratings  by  about  two-thirds  of  the  respondents.  However, 
the  remaining  topic  in  this  group,  Negotiating  Contracts,  was  given  the 
lowest  usefulness  rating  of  any  in  this  or  the  other  recurrent  courses. 

Only  21  percent  of  the  respondents  rated  it  as  of  even  moderate  utility.  A 
total  of  37  respondents  rated  this  topic  as  having  no  value  at  all. 


j.  Contract  Negotiations.  Following  the  trend  noted  in  the 
general  discussion  of  contract  negotiation  presented  in  the  previous  block, 
the  topics  in  this  block  were  felt  to  be  of  very  limited  value  by  graduates. 
The  majority  felt  each  of  the  three  topics  discussed  was  of  no  or  slight  use, 


although  almost  half  (47  percent)  of  the  respondents  felt  that  consideration  of 
Post-Negotiation  Responsibilities  had  been  at  least  moderately  helpful. 

Only  a third  felt  similarly  about  the  presentations  on  the  Negotiation 
Process  (37  percent)  and  Actions  Prior  to  Negotiation  (35  percent). 
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3.  The  LMR  Course  and  Supervisory  Behavior. 


a.  Self-Assessment  (N  = 104). 

(1)  Most  important  changes.  According  to  the  respondents, 
the  most  often  mentioned  change  (54  percent)  resulting  from  attendance  at 
the  LMR  course  was  some  general  or  specific  increment  in  knowledge  of  and 
ability  to  handle  LMR  situations  (Table  13).  The  most  frequently  mentioned 
specific  change  was  in  the  area  of  contract  administration  (13  percent); 
however,  nearly  all  of  these  citations  (19  of  22)  were  made  by  AT  supervisors. 
AF  supervisors  were  more  likely  to  cite  general  considerations,  such  as 
understanding  LMR,  human  relations,  and  improved  general  supervisory  skills. 
This  difference  suggests  that  contract  administration  is  a more  urgent 
consideration  of  AT  supervisors  at  this  point.  It  should  be  noted  that  with 
the  implementation  of  the  new  union  contract  in  the  AF  service,  this  aspect 
of  the  LMR  course  is  likely  to  assume  greater  significance  for  the  AF 
supervisor. 

The  other  positive  changes  mentioned  with  some  frequency  concerned 
understanding  human  relations  (11  percent)  and  personal  growth  (7  percent). 
There  were  15  respondents  (9  percent)  who  felt  that  there  had  been  no 
change  in  their  supervisory  behavior  as  a result  of  attending  the  course. 


TABLE  13.  Stannary  of  Statements  Concerning  Specific  Changes  In  Supervisory  Behavior 
Resulting  From  Attendance  at  LMR  Course 


• 

Number 

of  Responses  by  Service3 

Totals 

Type  of  Statement 

AF 

(N=35) 

AT 

(N=57) 

FS 

(N=l ) 

0 

(N=9 ) 

Number  of 
Responses 

Percent 
of  Total5 

Increased  knowledge  of  specific  application 

17 

65 

0 

9 

91 

54 

of  LMR  program  to  supervisory  situations 

Contract  adml nl stratlon 

1 

19 

0 

Z 

ZZ 

13 

Grievances 

1 

8 

0 

1 

10 

6 

Relations  with  unions 

4 

3 

0 

3 

10 

6 

Other 

11 

35 

0 

3 

49 

30 

Improved  general  supervisory  skills 

10 

11 

0 

1 

22 

13 

Increased  understanding  of  and  Improved 

11 

8 

0 

0 

19 

11 

hup  in  relations 

No  changes  reported 

8 

6 

1 

0 

15 

9 

Personal  growth 

4 

7 

0 

0 

11 

7 

Miscellaneous 

2 

9 

0 

0 

11 

7 

»AF  - Almay  Facilities  bSta»  of  percentages  may  vary  slightly  from  total  due  to  rounding  error. 

AT  - Air  Traffic 
FS  - Flltfit  Standards 
0 - Other 
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TABLE  14.  Percentages  of  Respondents  in  Graduate,  Inaedlate  Stperior,  and  Sipervlsee 
Reporting  Change  in  S*4>ervisory  Behavior  of  Graduates  of  Lffi  Course 


is  are  listed  as  presented  in  questionnaire  provided  to  graduates.  See  Appendi*es  for  corresponding  wordings  used  in  tl 
itionnalres  given  to  stgwriors  and  sigservisees.  there  no  corresponding  itee  was  used,  a dash  is  inserted  in  the  table. 


(2)  Behavioral  ratings.  There  were  45  specific  behavioral 
outcomes  specified  for  the  LMR  course  (Table  14).  Averaging  across  all  these 
outcomes,  it  was  found  that  78.5  percent  of  the  respondents  reported  at  least 
some  improvement  in  performance  on  LMR  matters  as  a result  of  attending  this 
course.  This  was  the  highest  value  for  the  four  courses  and  differed 
significantly  (£  < .05)2  from  each  of  the  others.  On  only  one  area  did  as 
many  as  five  respondents  report  lessened  effectiveness,  and  the  average  was 
less  than  one  respondent  (.6)  per  item. 

More  than  half  (23)  the  behavioral  items  were  rated  as  showing  some  or 
more  improvement  by  80  percent  or  more  of  the  respondents.  Three  of  these, 
understanding  the  executive  order  on  LMR  (#1),  knowing  procedures  to  be  used 
in  meeting  with  labor  organizations  (#15),  and  understanding  the  role  of  the 
supervisor  in  administering  a labor  contract  (#21)  were  judged  improved  by 
more  than  90  percent  of  the  graduates. 

Of  the  23  areas  cited  above,  9 concerned  the  grievance  process  (#29,  #31- 
38).  Thus,  not  only  was  this  area  of  teaching  felt  to  be  most  useful  by 
graduates  (see  previous  discussion),  it  was  also  the  area  in  which  they  felt 
the  most  improvement  in  performance. 

The  two  areas  of  performance  that  showed  least  improvement  according  to 
the  graduates  were  awareness  of  indications  of  the  effectiveness  of  a labor 
contract  (#24),  and  ability  to  prepare  a grievance  case  for  arbitration  (#44). 
Just  about  half  (47  and  50  percent  respectively  for  the  two  items)  of  the 
graduates  reported  improved  performance  in  these  two  areas. 


There  was  only  one  area  of  the  ability  to  resolve  ULPs  in  a satisfactory 
manner  (#28)  that  was  rated  differentially  by  the  groups  participating  in 
the  survey.  On  this  item,  75.0  percent  of  the  HD  and  76.7  percent  of  the  FF 
respondents,  but  only  30.0  percent  of  the  RO  group,  reported  improved 
functioning,  a difference  statistically  significant  by  chi-square  test 
(£  < .01).  It  is  not  clear  why  the  RO  graduates  were  less  likely  to  report 
improvement  in  this  area.  Perhaps  being  at  the  middle  level  of  the  management 
structure  provides  less  opportunity  to  deal  effectively  with  ULP  matters  that 
are  brought  to  the  RO.  However,  it  should  again  be  noted  that  the  number  of 
RO  graduates  involved  in  these  comparisons  was  relatively  small  and  the 
findings  should  therefore  be  viewed  as  only  suggestive. 


immediate 
ates  than 
superiors 
As  it  was 
the  other 


b.  Assessment  by  Immediate  Superiors  (N  = 55).  As  a group, 
superiors  reported  only  slightly  less  behavioral  change  in  gradu- 
did  the  graduates  themselves.  The  average  proportion  of  immediate 
indicating  improvement  on  any  item  was  75.5  percent  (Table  14). 
for  graduates,  this  value  was  significantly  higher  than  those  for 
three  courses  (£  < .05)^. 


There  were  six  areas  in  which  more  than  90  percent  of  the  immediate 
superiors  rated  graduates  improved  after  the  LMR  course;  understanding  of  the 
Executive  Order  on  LMR  in  government  (#1),  awareness  of  communication  require- 


26 


merits  associated  with  the  presence  of  labor  organizations  (#8),  recognition  of 
problem  areas  with  union  relations  (#22),  understanding  the  differences 
between  grievances,  unfair  labor  practices,  and  appeals  (#29),  knowledge  of 
representation  rights  of  employees  and  unions  in  grievance  actions  (#31),  and 
awareness  of  responsibility  for  notification  of  unions  in  grievance  actions 
(#36). 

The  behavioral  areas  that  were  least  likely  to  be  rated  as  improved 
concerned  interpretation  of  contracts  (#10),  contract  negotiations  (#18,19,20), 
evaluation  of  contract  effectiveness  (#24),  and  arbitration  (#41,44).  Only 
on  awareness  of  negotiating  techniques  did  less  than  half  of  the  ratings 
reflect  improvement  (#19). 

There  were  only  six  behavioral  items  to  which  immediate  superiors  reported 
diminished  functioning  in  graduates,  and  only  one  such  rating  was  made  in  each 
case.  Four  of  these  items  concerned  the  performance  of  a single  individual, 
and  the  remaining  two  items  involved  a second  person. 

There  were  no  differences  in  ratings  as  a function  of  location  or  FAA 
service . 


c.  Assessment  by  Supervisees  (N  = 70).  The  supervisees  who 
responded  (Thble  14)  reported  significantly  less  change  in  graduates  than 
either  the  graduates  themselves  (£  < .01)2  or  superiors  (jd  < .01)2.  However, 
the  average  percentage  of  60.6  of  the  supervisees  reporting  improved  perform- 
ance in  the  LMR  areas  was  significantly  higher  than  the  parallel  judgments 
made  on  the  effectiveness  of  the  other  three  recurrent  courses  (j>  < .01  for 

each  comparison)2. 

Ratings  by  supervisees  of  lessened  effectiveness  in  the  LMR  area  by 
graduates  were  relatively  rare.  An  average  of  1.3  supervisees  per  item 
reported  reduced  effectiveness,  and  on  only  the  item  concerning  resolution  of 
ULPs  (#28)  did  as  many  as  four  persons  give  such  ratings. 

The  four  behavioral  items  with  the  highest  percentage  of  supervisees 
rating  their  supervisors  as  improved  were  understanding  of  the  impact  of 
unionization  on  operations  (#7),  awareness  of  problems  associated  with  the 
presence  of  unions  (#9),  knowledge  of  procedures  for  handling  unfair  labor 
practices  (#13) , and  understanding  of  differences  between  complaints, 
grievances,  unfair  labor  practices,  and  appeals  (#29).  Approximately  two-thirds 
of  the  respondents  reported  improvements  in  these  areas. 

The  least  improvement  was  noted  in  the  appreciation  of  the  role  of  the 
union  representative  in  LMR  (#23)  and  ability  to  resolve  unfair  labor 
practices  (#28).  Still,  more  than  50  percent  of  the  respondents  noted  some 
improvement  in  each  of  these  areas.  With  respect  to  the  latter  area,  four 
supervisees  noted  decreased  effectiveness;  this  was  the  only  case  for  which 
more  than  two  such  ratings  were  made. 


There  were  no  differences  in  ratings  between  FAA  services  or  the 
different  locations. 

d.  Comparing  the  Three  Sets  of  Ratings.  Graduates  and  their 
immediate  superiors  were  more  likely  than  supervisees  to  report  improved 
supervisory  functioning  in  the  LMR  area.  As  noted  earlier,  an  average  of 
about  three-fourths  of  the  graduates  and  immediate  superiors  reported 
behavioral  change  on  these  items.  These  were  the  highest  values  for  any  of 
the  four  courses.  Also  at  a high  value  compared  to  the  other  courses  was  the 
average  frequency  of  60  percent  at  which  the  supervisees  reported  increased 
effectiveness.  This  value  was,  however,  significantly  lower  than  the 
percentages  obtained  from  graduates  and  superiors  in  this  course  (£  < .01  for 
both  comparisons )2 . 

Rankings  of  the  items  by  percentage  of  respondents  reporting  behavior 
change  for  each  of  the  three  groups  and  correlational  analyses  of  those 
rankings  showed  a high  degree  of  relationship  between  graduates  and  their 
immediate  superiors  (rs  = .86,  £ < .01).  There  was  no  such  relationship 
between  rankings  by  graduates  and  supervisees  (rs  = .31,  not  significant). 

As  the  correlations  would  suggest,  there  were  more  instances  in  which 
graduates  and  immediate  superiors  most  frequently  or  least  frequently  reported 
improved  performance  in  the  same  areas  than  was  true  for  graduates  and  super- 
visees. Among  the  areas  most  frequently  rated  as  improved,  both  graduates  and 
their  immediate  superiors  tended  more  often  to  report  improvement  by  the 
graduate  in  understanding  of  the  Executive  Order  governing  LMR,  awareness  of 
communication  requirements  associated  with  union  presence,  recognition  of 
potential  LMR  problem,  understanding  of  the  differences  between  complaints, 
grievances,  and  ULPs , awareness  of  the  grievance  system,  and  awareness  of 
requirements  to  inform  unions  of  grievance  procedures. 

At  the  low  end  of  the  rankings,  both  graduates  and  their  immediate 
superiors  were  less  likely  to  report  change  in  knowledge  of  indicators  of 
effective  labor  contracts  or  knowledge  of  how  to  prepare  a grievance  for 
arbitration. 

Graduates  and  supervisees  agreed  in  seeing  more  frequent  improvement  in 
understanding  of  the  impact  of  unions  and  awareness  of  problems  associated 
with  the  presence  of  unions  in  the  work  setting.  Both  groups  also  noted 
improvement  in  the  supervisor's  knowledge  of  requirements  for  meeting  with 
unions  on  work  and  employee  matters  relatively  frequently. 

4.  Impact  on  the  Unit. 

The  impact  of  the  LMR  course  on  conditions  in  the  supervisor's 
unit  was  comparably  rated  in  general  by  both  graduates  and  supervisees  (Table 
15).  As  with  the  other  three  courses,  the  most  favorable  ratings  were  given 
to  the  employee-supervisor  interaction  followed  in  order  by  efficiency,  job 
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satisfaction,  and  employee  morale.  The  only  notable  difference  in  ratings 
between  the  graduates  and  supervisees  concerned  the  interaction  between  them 
as  20  percent  more  of  the  graduates  felt  an  improvement  in  this  area  had 
occurred. 

Ratings  indicating  that  conditions  in  the  unit  had  worsened  after  the 
graduate's  attendance  at  the  LMR  course  were  rare.  Two  graduates  and  four 
supervisees  felt  job  satisfaction  had  decreased.  Three  graduates  and  six 
supervisees  reported  lowered  morale. 


TABLE  15.  Percentage  of  Craduates  and  Supervisees  Rating  Conditions  In 
Their  Unit  as  IngJroved  After  Attendance  of  Graduate  at  LMR  Course 


Item 

Graduate 

(N=104) 

Supervisee 

(N=70) 

Unit  Efficiency 

48.1 

40.3 

Employee  Job  Satisfaction 

37.5 

34.7 

Employee  Morale 

36.5 

32.4 

Sipervisor's  Interaction  With  Enployees 

66.3 

46.3 

There  were  significant  (£  < .05)  differences  according  to  chi-square 
analysis  between  the  responses  of  graduates  from  the  FAA  services  on  three 
of  the  four  items  concerning  unit  conditions.  Unit  efficiency  was  rated 
improved  by  61  percent  of  the  AT,  34  percent  of  the  AF,  and  only  20  percent 
of  the  FS/0  graduates  of  this  course.  The  corresponding  values  for  improved 
job  satisfaction  were  51  percent,  17  percent,  and  30  percent,  and  for 
improved  unit  morale  they  were  44  percent,  20  percent,  and  20  percent.  In 
each  case,  graduates  from  the  AT  service  were  much  more  likely  to  report 
improvement  than  either  the  AF  or  FS/0  graduates.  However,  no  parallel  trend 
was  noted  in  the  supervisee  data. 

C.  CD  Course. 

1.  General  Evaluation  by  MTS  Graduates  (N  = 102). 

a.  Open-Ended  Questionnaire.  The  most  helpful  aspects  of  the  CD 
course  are  detailed  in  Table  16.  The  respondents  most  frequently  mentioned 
some  area  in  which  they  had  gained  additional  knowledge  and  understanding  of 
CD  as  being  most  helpful.  In  this  respect,  many  (21  percent)  specifically 
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indicated  that  their  increased  understanding  of  how  to  proceed  in  undertaking 
CD  actions  was  most  helpful.  Another  substantial  group  (19  percent)  felt  that 
the  discussion  of  the  philosophy  and  purposes  of  CD  had  been  especially 
useful.  Other  helpful  aspects  of  the  course  mentioned  by  at  least  10  percent 
of  the  respondents  included  working  through  problems  with  others,  an  increased 
understanding  of  the  supervisory  role,  and  personal  growth. 


TABLE  16.  Suwaary  of  Statenents  Indicating  the  Most  Helpful  Aspects  of  the  CO  Course 


Type  of  Statemnt 

Ntntoer  of  Responses  by  Servloe* 

Totals 

AF 

(N»36> 

AT 

<N*42) 

FS 

(N*16) 

0 

(M-9) 

Nuatoer  of 
Responses 

Percent 
of  Total 

Increased  Knowledge  end  Understanding 

52 

74 

24 

15 

165 

79 

CO  Procedures  end  A (Ministration 

15 

21 

5 

4 

45 

21 

Purpose  end  Philosophy  of  CD 

15 

1% 

6 

5 

40 

19 

Sqpervisory  Role 

3 

15 

3 

0 

21 

10 

General  Principles  of  Stpervlslon 

7 

6 

1 

4 

18 

9 

Recognition  of  Behavior  Proto lews  In  Employees 

5 

6 

4 

1 

16 

8 

Understanding  Ej^loyees/Huaan  Relations 

A 

4 

2 

0 

10 

5 

Other 

3 

8 

3 

1 

15 

7 

Working  Through  Supervisory  Protolean  With  Others 

6 

11 

4 

1 

22 

10 

Personal  Growth 

7 

8 

5 

0 

20 

10 

Miscellaneous 

2 

1 

0 

0 

3 

1 

*A F - Airway  Facilities 
AT  - Air  Traffic 
FS  - Fllgfit  Standards 
0 - Other 


TABLE  17.  Summary  of  Statements  Indicating  Areas  of  Needed  Improvement  In  the  CD  Course 


Number  of  Responses  by  Service4  Totals 


Type  of  Statement 

AF 

<N=3«) 

AT 

(N=42) 

FS 

(N=16) 

0 

(N=9) 

Number  of 
Responses 

Percent 
of  Total" 

Course  Content 

14 

28 

8 

0 

50 

36 

Modify  Course  Approach 

5 

12 

6 

5 

28 

20 

More  Practical  Emphasis 

4 

9 

4 

2 

19 

14 

Other 

1 

3 

2 

3 

9 

6 

Leave  Course  As  Is 

7 

5 

3 

4 

19 

14 

Miscellaneous 

15 

18 

7 

3 

43 

31 

* AF  - Airway  Facilities  b Sum  of  percentages  may  vary  slightly  from  total 

AT  - Air  Traffic  due  to  rounding  error. 

FS  - Flight  Standards 
0 - Other 
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As  with  the  other  courses,  no  one  aspect  of  this  course  was  singled  out  as 
particularly  in  need  of  change  (Table  17).  The  greatest  percentage  of  comments 
related  to  some  aspect  of  the  content  of  the  course  (36  percent);  however,  no 
one  specific  area  of  course  content  was  mentioned  by  as  many  as  10  percent  of 
the  respondents. 

Outside  of  course  content,  the  area  most  often  mentioned  as  needing 
improvement  was  that  of  a more  practical  emphasis  in  the  presentation  (14 
percent).  An  equal  number  of  respondents  recommended  that  no  changes  be  made 
in  the  course. 

These  response  patterns  suggest  that  no  particular  aspect  of  the  course 
was  considered  seriously  deficient. 

b.  Usefulness  of  the  CD  Course.  The  course  was  rated  moder- 
ately or  more  useful  by  94.0  percent  of  the  graduates.  All  respondents  rated 
the  course  as  having  some  value. 

2.  Evaluation  of  Course  Content. 

The  average  proportion  of  respondents  rating  each  topic  as  useful 
was  74.8  percent.  This  value  is  somewhat  higher  than  that  obtained  for  the 
PIP/PER  and  LMR  courses  and  about  the  same  as  the  value  for  the  ME  course.  In 
other  words,  there  is  some  tendency  (statistically  significant  at  £ < .05)^ 
for  the  graduates  to  rate  the  content  of  this  course  as  more  useful  than  was 
the  case  for  the  two  courses  already  described. 

The  pattern  of  extreme  ratings  showed  that  graduates  were  more  likely  to 
evaluate  the  course  topics  in  highly  positive  than  in  particularly  negative 
terms.  Their  number  of  "extremely  useful"  ratings  averaged  between  6 and  7 per 
topic,  while  the  number  of  ratings  of  "not  at  all  useful"  numbered  just  4 on 
the  average. 

There  were  no  topics  in  the  CD  course  that  were  rated  differently  by  gradu- 
ates from  different  services  or  locations. 

a.  Meaning  and  Purpose  of  Discipline.  Each  of  the  four  specific 
topics  taught  under  this  unit  were  rated  as  quite  useful  by  the  graduates,  as 
no  less  than  85  percent  indicated  the  areas  were  of  moderate  or  better  utility 
(Table  18).  The  topic  of  the  Supervisor's  Role  in  Preventing  Need  for  Correc- 
tive Discipline  was  particularly  well  regarded  (89  percent)  as  55  of  1§2 
ratings  were  "very  useful"  and  another  17  were  "extremely  useful."  There  were 
also  17  "extremely  useful"  ratings  for  the  subject  of  Positive  versus  Negative 
Discipline  and  13  for  Identifying  Potential  Problem  Areas.  Only  one  other 
topic,  the  Supervisor's  Role  in  Discipline  (see  (3))  was  so  highly  rated  by 
as  many  as  10  participants. 
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TABLE  18.  Usefulness  Ratings  of  CO  Course  Content 


Content  Area 

Percent  rating 
Moderate,  quite, 
or  very  useful 

Meaning  and  Purpose  of  Discipline 

Role  of  Supervisor  In  Preventing  Need 

86.9 

for  Corrective  Discipline 

89.2 

Identifying  Potential  Problem  Areas 

87.2 

Concepts  of  Positive  vs.  Negative  Discipline 

86.0 

Relationship  of  Discipline  to  Employee  Motivation 

85.1 

Informal  Discipline 

81.2 

Available  Disciplinary  Actions 

83.2 

Approaches  to  the  Disciplinary  Interview 

82.2 

How  to  Employ  Informal  Discipline 

78.4 

FAA  Policy  on  Discipline 

79.0 

Role  of  Supervisor  In  Discipline 

90.2 

FAA  Standards  of  Conduct 

79.4 

FAA  Philosophy  on  Discipline 

76.2 

Relation  of  Discipline  to  Other  FAA  Programs 

70.2 

Formal  Discipline 

77.3 

Letter  of  Reprimand 

80.0 

Employee  Appeals  System 

77.2 

Minor  Adverse  Actions 

77.0 

Major  Adverse  Actions 

75.0 

Human  Relations 

76.1 

Role  of  Supervisors  In  Human  Relations 

Impact  of  Organizational  Environment  on 

81.8 

Employee  Behavior 

76.3 

Review  of  Motivational  Theory 

70.2 

Investigations  and  Security 

63.7 

Role  of  Supervisor  in  Investigative  Actions 

67.4 

Investigative  and  Security  Resources 

60.0 

Ethical  Conduct 

62.7 

Supervisory  Responsibilities 

78.4 

Outside  Employment/Financial  Interests 

55.9 

Clfts,  Favors,  Entertainment 

53.9 

Other  Behavioral  Problems 

62.6 

Financial  Obligations 

67.3 

Alcohol  Abuse 

63.7 

Political  Activities 

56.9 
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b.  Informal  Discipline.  Two  of  the  three  topics  in  this  block, 
Available  Discipline  Actions  (83  percent)  and  Approaches  to  the  Disciplinary 
Interview  (82  percent),  were  rated  as  moderately  or  more  useful  by  more  than 
80  percent  of  the  graduates.  The  other  area,  How  to  Employ  Informal  Actions, 
was  so  judged  by  78  percent  of  the  respondents. 

c.  FAA  Policy  for  Discipline.  One  of  the  specific  topics  in  this 
unit,  the  Role  of  the  Supervisor  in  Discipline,  received  the  highest  of  all 
the  ratings  for  this  course;  90  percent  of  the  graduates  endorsed  its  utility 
at  the  moderate  level  or  better.  The  topics  of  FAA  Standards  of  Conduct  (79 
percent)  and  FAA  Philosophy  of  Discipline  (76  percent)  although  not  rated  as 
high,  were  still  rated  at  the  moderately  or  more  useful  level  by  more  than 
three-fourths  of  the  respondents.  The  lowest  rated  topic  in  this  block  was 
the  presentation  on  Discipline  in  Relation  to  Other  FAA  Programs,  but  still 

70  percent  of  the  respondents  felt  it  was  at  least  moderately  useful. 

d.  Formal  Discipline.  The  four  topics  in  this  block  were  rated 
similarly,  as  between  75  percent  and  80  percent  of  the  respondents  felt  at 
least  moderate  utility  in  these  subject  areas.  The  area  receiving  the  highest 
ratings  concerned  Letters  of  Reprimand  (80  percent). 

e.  Human  Relations.  The  ratings  for  the  three  areas  considered 
in  this  instruction  block  varied  considerably.  Almost  82  percent  of  the 
graduates  rated  the  discussion  of  the  Supervisor's  Role  in  Human  Relations  to 
be  of  moderate  or  better  utility.  Next,  76  percent  rated  the  presentation  on 
the  Impact  of  the  Organization  on  Employee  Behavior  as  at  least  "moderately 
useful."  About  70  percent  of  the  graduates  felt  the  Review  of  Motivational 
Theory  was  moderately  or  more  useful. 

f.  Investigations  and  Security.  Compared  to  the  five  instruc- 
tional blocks  discussed  above,  this  block  and  the  remaining  two  blocks  were 
judged  considerably  less  useful  by  respondents.  Just  over  two-thirds  of  the 
graduates  rated  the  topic  of  the  Supervisor's  Role  in  Investigative  Actions  as 
moderately  or  more  useful,  while  60  percent  so  rated  the  presentation  on  the 
Investigation  and  Security  Resources  Available  to  Supervisors. 

g.  Ethical  Conduct.  One  of  the  three  areas  in  this  block  was 
rated  very  favorably;  the  presentation  on  the  Supervisor's  Responsibilities 
for  Ethical  Conduct  was  rated  moderately  or  more  useful  by  78  percent  of  the 
graduates.  The  other  two  areas,  Outside  Financial  Interests  and  that  of 
Gifts,  Favors,  and  Entertainment  were  the  lowest  rated  of  all  topics  in  the 
course  with  only  56  percent  and  54  percent  of  the  respondents  reporting 
moderate  or  more  utility  respectively. 

h.  Other  Behavioral  Problems.  The  discussions  of  Financial 
Obligations  (67  percent)  and  Alcohol  Abuse  (64  percent)  were  seen  as  at  least 
moderately  useful  by  about  two-thirds  of  the  graduates.  The  topic  of  Political 
Activities  was  given  such  ratings  by  57  percent  of  the  respondents. 
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3.  The  CD  Course  and  Supervisory  Behavior, 
a.  Self-Assessment  (N  = 102). 

(1)  Most  important  changes.  Most  of  the  comments  (60  percent) 
about  behavioral  changes  concerned  increased  knowledge  and  understanding  of 
the  application  of  CD  to  the  work  situation  (Table  19).  Specific  change's 
mentioned  under  this  general  classification  included  increased  ability  to 
solve  discipline  problems  (21  percent),  improved  discipline  procedures  (19 
percent),  and  an  increased  understanding  of  the  philosophy  of  CD  (15  percent). 


TABLE  19.  S unitary  of  Statements  Concerning  Specific  Changes  In  Supervisory  Behavior 


Resulting  From  Attendance  at  CD  Course 


Number  of  Responses  by  Service2 3 

Totals 

Type  of  Statement 

AF 

(N=3 6) 

AT 

<N=4 2) 

FS 

(N = 16) 

0 

(N=9) 

Number  of 
Responses 

Percent 
of  Total” 

Increased  knowledge  of  specific  applied* Ion  of 

30 

41 

13 

6 

90 

60 

CD  program  to  supervisory  situations 

Ability  to  solve  discipline  problems 

12 

15 

4 

1 

CD  procedures 

8 

15 

4 

2 

29 

19 

Understanding  of  CD  philosophy 

6 

10 

4 

3 

23 

15 

Other 

4 

1 

1 

0 

6 

Personal  growth 

5 

12 

3 

1 

21 

14 

Mo  changes  reported 

7 

5 

1 

4 

17 

11 

Increased  understanding  of  and  Improved 

1 

6 

2 

2 

11 

7 

human  relations 

Improved  general  supervisory  skills 

3 

4 

3 

0 

10 

7 

Mlscel laneous 

2 

2 

l 

0 

5 

3 

dAF  - Airway  Facilities  bSun  of  percentages  may  vary  slightly  from  total  due  to  rounding  error. 

AT  - Air  Traffic 
FS  - Elicit  Standards 
0 - Other 


Other  areas  of  improvement  noted  personal  growth  (14  percent),  improved 
understanding  of  and  relations  with  employees  (7  percent),  and  improvement  in 
general  supervisory  skills  (7  percent).  There  were  17  respondents  (11 
percent)  who  reported  no  behavioral  changes  as  a function  of  attending  this 
course . 


(2)  Behavioral  ratings.  On  the  average,  for  the  34  behaviors 
considered,  72.8  percent  of  the  CD  graduates  felt  that  performance  had  improved 

(Table  20).  However,  there  was  some  greater  tendency  to  report  diminished 
performance  after  attendance  at  the  CD  course  than  was  true  for  the  other 
three  courses.  An  average  of  3.4  respondents  per  item  indicated  a loss  in 
effectiveness  with  as  many  as  seven  and  never  less  than  one  respondent 
reporting  reduced  performance. 
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*re  listed  as  presented  In  questionnaire  provided  to  graduates.  See  Appendixes  for  corresponding  wordlnqs  used  In  the  questionnaires  given  to 
iors  and  supervisees,  there  no  corresponding  Item  was  used,  a dash  is  Inserted  in  the  table. 


There  were  two  areas  of  performance,  knowledge  of  informal  actions  (#17), 
and  awareness  of  documentation  and  administrative  requirements  for  disciplinary 
action  (#23),  that  were  rated  as  improved  by  more  than  80  percent  of  the 
respondents.  Another  eight  factors  were  rated  as  improved  by  more  than  73 
percent  of  the  graduates,  but  these  did  not  cluster  in  any  particular 
behavioral  area.  They  included  understanding  employee  motivation  (#1),  under- 
standing supervisory  responsibilities  for  discipline  in  general  (#4), 
recognition  of  problems  (#14),  understanding  informal  discipline  procedures 
(#18),  ability  to  conduct  an  effective  disciplinary  interview  (#19),  knowing 
how  to  take  minor  adverse  actions  (#24),  preparation  of  letters  proposing 
adverse  action  (#27),  and  awareness  of  supervisory  responsibilities  for 
certain  behavioral  problems  such  as  alcoholism  (#30). 

lhc  three  behavioral  items  least  likely  to  be  rated  as  improved  were 
understanding  the  relation  of  discipline  to  other  FAA  programs  (#5),  under- 
standing the  relation  of  discipline  to  organizational  goals  (#9),  and  use  of 
invr  t igat  ive/security  services  (#34).  However,  in  each  case  at  least  50 
p<  t nt  of  trie  respondents  did  report  increased  effectiveness  in  themselves. 

There  was  one  item,  awareness  of  the  appeal  process  (#25),  for  which  -he 
ratings  differed  across  the  FAA  services  according  to  chi-square  test  < 

.05).  In  this  case,  80.0  percent  of  the  AF  and  FS/0  graduates  but  only  52.4 
percent  of  the  AT  graduates  reported  improved  functioning. 

There  were  no  differences  in  behavioral  ratings  as  a function  of  location 
of  the  graduate. 

b.  Assessment  by  Immediate  Superiors  (N  = 69). 

Immediate  superiors  less  frequently  observed  behavioral  change 
in  graduates  than  graduates  did  themselves  (Table  20).  The  difference 
between  the  mean  rating  of  immediate  superiors  at  65.9  percent  and  graduates 
at  72.8  percent  was  significant  (£  < .01)2. 

The  areas  in  which  immediate  superiors  most  frequently  noted  improvement 
were  in  understanding  management  style  (#2),  awareness  of  FAA  policy  and 
ptii losophy  of  discipline  (#3),  knowledge  of  informal  actions  (#17),  and 
awareness  of  documentation  and  administrative  requirements  (#23).  Each  of 
these  behavioral  areas  was  rated  as  improved  by  more  than  three-fourths  of 
the  respondents. 

The  two  factors  least  likely  to  be  seen  as  improved  were  the  correct  uses 
of  the  letter  of  reprimand  (#20)  and  appropriate  use  of  investigative/security 
services  (#34).  In  each  case  less  than  50  percent  of  the  superiors  felt 
that  the  graduates  had  shown  improved  performance. 


Diminished  performance  in  graduates  was  noted  by  at  least  one  immediate 
superior  for  all  but  seven  areas;  however,  no  more  than  two  such  ratings  were 
made  for  any  one  area. 
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There  was  a cluster  of  three  behavioral  items  concerning  adverse  actions 
for  which  the  frequency  of  improvements  noted  in  the  performance  of  graduates 
differed  for  immediate  superiors  from  the  various  FAA  services  according  to 
chi-square  analysis.  Both  AT  (77.8  percent)  and  AF  (65.0  percent)  superiors 
more  frequently  reported  improvement  (£  < .05)  on  ability  to  prepare  a 
proposal  letter  for  an  adverse  action  (#27)  than  FS/u  (35.3  percent)  superiors. 
Likewise,  AT  (74.1  percent)  and  AF  (65.0  percent)  graduates  were  also  more 
likely  to  be  judged  (j>  < .01)  by  their  immediate  superiors  as  having  improved 
in  their  understanding  of  how  to  take  a major  adverse  action  (#28)  than  was 
the  case  for  the  ratings  by  the  immediate  superiors  of  FS/0  graduates  (25.0 
percent).  This  pattern  was  also  present  concerning  understanding  of  the  appeal 
process.  For  a major  adverse  action  (#29),  71.4  percent  of  the  AF , 66.7 
percent  of  the  AT,  and  31.2  percent  of  the  FS/0  superiors  reported  improved 
functioning  on  the  part  of  graduates  (j>  < .05).  This  pattern  suggests  that 
the  segment  of  the  course  concerned  with  these  processes  had  little  impact  on 
graduates  outside  of  the  two  largest  FAA  services. 

c.  Assessment  by  Supervisees  (N  = 64). 

As  in  the  other  courses,  supervisees  less  frequently  reported 
change  (Table  20)  than  either  graduates  (£  < .01)2  or  their  superiors  (£  < 
.01)2.  The  average  proportion  of  supervisees  noting  improved  performance  was 
48.4  percent. 

Diminished  performance  by  graduates  was  reported  by  two  or  more  supervisees 
on  each  of  the  behavioral  items  and  averaged  3.75  such  ratings  per  item.  This 
frequency  is  substantially  higher  than  the  corresponding  values  for  the  two 
courses  already  discussed  but  not  as  high  as  that  for  the  ME  course.  The 
highest  number  of  such  ratings  (seven)  was  given  to  the  item  concerning  ability 
to  identify  causes  of  employee  problems  (#12).  Since  this  area  of  performance 
was  also  rated  as  improved  in  a substantial  number  of  cases,  it  suggests  that 
the  courfte  may  have  had  some  effect  in  both  directions,  depending  on  the 
individual  supervisor  being  trained. 

For  half  of  the  performance  areas,  at  least  50  percent  of  the  supervisees 
reported  improved  performance  by  graduates.  The  behavior  most  frequently 
reported  as  improved  was  the  implementation  of  practices  to  prevent  need  for 
corrective  discipline  (#11).  Three  other  areas  of  functioning,  encouragement 
of  self-discipline  (#10),  recognition  of  discipline  problems  (#14),  and 
awareness  of  causes  of  deterioration  in  discipline  (#15)  were  also  judged  to 
have  improved  rather  frequently. 

Only  a third  of  the  supervisees  reported  that  graduates  were  more  effective 
in  taking  appropriate  action  to  determine  causes  of  deterioration  in 
discipline  (#16)  or  in  relating  discipline  to  organizational  goals  (#9). 

There  were  no  differences  in  the  response  tendencies  of  supervisees  in 
different  locations;  however,  AT  respondents  did  differ  from  FS/0  supervisees 
in  their  judgments  on  two  areas  and  from  AF  supervisees  on  one  performance 
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area.  The  behavior  on  which  the  judgments  of  the  AT  group  differed  from  both 
of  the  other  groups  by  chi-square  test  (]3  < .05)  concerned  the  implementation 
of  supervisory  practices  to  prevent  need  for  corrective  discipline  (#11).  A 
total  of  85.0  percent  of  those  in  AT,  52.4  percent  in  AF  and  29.4  percent  of 
the  FS/0  group  indicated  that  some  improvement  had  occurred  in  this  area. 

With  respect  to  viewing  discipline  as  a positive  factor  instead  of  a means  of 
punishment  (#6),  75.0  percent  of  the  AT,  50.0  percent  of  the  AF,  and  26.3 
percent  of  the  FS/0  supervisees  reported  improvement  by  their  supervisors. 

In  this  case  only  the  difference  between  the  AT  group  and  the  FS/0  group  was 
significant  (£  < .05). 

d.  Comparing  the  Three  Sets  of  Ratings. 

As  usual,  graduates  were  most  likely  to  report  improved 
effectiveness  (73  percent)  than  were  immediate  superiors  (66  percent,  £ < 

.05)2  anc]  supervisees  (48  percent,  £ < .01)2. 

The  correlation  between  the  rankings  of  the  frequencies  of  reported 
behavioral  change  was  negligible  between  graduates  and  either  immediate 
superiors  (rs  = .23)  or  supervisees  (rs  = .01).  In  other  words,  there  was  no 
notable  correspondence  between  the  groups  in  the  relative  frequency  with  which 
they  reported  improved  in  supervisory  behaviors. 

The  three  groups  did  not  unanimously  agree  on  any  of  the  most  improved  or 
least  improved  areas  of  functioning.  Graduates  and  their  immediate  superiors 
tended  to  see  improvement  more  frequently  in  awareness  of  informal  approaches 
to  discipline  and  documentation  requirements  in  discipline.  Graduates  and 
supervisees  tended  to  feel  relatively  frequently  that  improvement  had 
occurred  in  recognition  of  the  symptoms  of  discipline  problems. 

With  respect  to  the  areas  of  least  change,  graduates  and  immediate 
superiors  tended  to  see  little  increased  effectiveness  in  the  use  of 
invest igative/security  services  within  the  agency,  while  graduates  and  super- 
visees saw  relatively  little  change  in  ability  to  relate  the  discipline 
program  to  FAA  organizational  goals. 

4.  Impact  on  the  Unit. 

As  for  the  other  courses,  both  graduates  and  supervisees  most 
frequently  noted  improvement  in  supervisor-employee  interactions,  followed  by 
improvements  in  efficiency,  job  satisfaction,  and  morale  (Table  21).  However, 
the  agreement  in  ratings  between  the  graduates  and  supervisees  was  somewhat 
less  than  for  the  other  three  courses. 

The  biggest  difference  was  on  the  item  concerning  interaction  with 
employees.  About  80  percent  of  the  graduates  felt  that  improvement  had 
occurred  in  this  area  but  not  quite  one-half  of  the  supervisees  felt  the  same 
way.  On  this  item  chi-square  analysis  revealed  that  there  was  a marked 
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TABLE  21.  Percentage  of  Graduates  and  Supervisees  Rating  Conditions  In 


Their  Unit  as  Improved  After  Attendance  of  Graduate  at  CD  Course 


Item 

Graduate 
(N=i02 ) 

Supervisee 

(Nr64) 

Unit  Efficiency 

59.4 

45.3 

Employee  3ob  Satisfaction 

38.6 

31.2 

Fmployee  Morale 

38.6 

29.7 

Supervisor's  Interaction  With  Employees 

80.2 

48.4 

difference  between  the  responses  of  supervisees  in  the  different  services  (p 
< .01).  Supervisees  in  the  AT  service  were  much  more  likely  (77.3  percent)  to 
report  improved  interactions  than  were  those  in  the  AF  (26.1  percent)  or  FS/O 
(42.1  percent)  services.  Closest  agreement  was  found  on  the  job  satisfaction 
and  employee  morale  items  where  about  a third  of  the  respondents  in  both 
groups  reported  some  improvement . 

Ratings  of  lessened  quality  in  unit  conditions  were  rare  for  both  graduates 
and  supervisees.  The  greatest  number  (6)  of  such  reports  were  made  by  super- 
visees on  the  morale  item,  otherwise  no  more  than  three  such  ratings  were 
given  to  any  item  by  supervisees  or  by  graduates. 

D.  ME  Course. 

1.  General  Evaluation  by  MTS  Graduates  (N  = 98). 

a.  Open-Ended  Questionnaire.  According  to  the  graduates  of  the  ME 
course,  the  most  helpful  aspect  of  the  course  was  the  opportunity  to  gain  addi- 
tional knowledge  and  understanding  of  various  aspects  of  management  (Table  22). 
Under  this  general  framework  frequent  mention  was  made  of  the  value  of  the 
review  of  general  management  principles,  human  relations,  and  team  action 
concepts.  There  were  also  several  statements  concerning  personal  growth  and 
the  opportunity  to  work  through  managerial  problems  with  others. 


TABLE  22.  Sumsary  of  Statements  Indicating  the  Host  Helpful  Aspects  of  the  HE  Course 


Type  of  Statement 

N usher 

of  Responses  by  Service* 

Totals 

AT 

(N-26) 

AT 

(N=35) 

FS 

(N=18) 

0 

( N= 18) 

Humber  of 
Responses 

Percent 
of  Totalb 

Increased  knowledge  and  Understanding 

28 

33 

29 

23 

113 

55 

General  Management  Principles 

10 

10 

7 

13 

40 

20 

Understanding  Employees /Human  Relations 

5 

6 

4 

3 

18 

9 

Team/Croup  Action 

3 

5 

7 

1 

16 

8 

Other 

10 

12 

11 

6 

39 

19 

Personal  Growth 

5 

15 

7 

8 

35 

17 

forking  Through  Management  Problems  With  Others 

7 

16 

4 

3 

30 

15 

Miscellaneous 

6 

8 

2 

11 

27 

13 

*A F - Airway  Facilities  bSua  of  percentages 

AT  - Air  Traffic 

may  vary  slightly  from  total  due 

to  rounding 

error . 

FS  - Flif^t  Standards 
0 - Other 
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There  was  no  particular  consensus  about  the  aspects  of  the  ME  course  most 
in  need  of  improvement  (Table  23).  The  general  area  of  course  content 
received  the  most  attention.  Within  the  course  content  category,  the  most 
frequent  recommendation  was  to  increase  the  emphasis  on  teaching  specific 
management  skills  such  as  Management  by  Objectives  (MBO)  or  budgeting.  There 
were  also  several  requests  for  more  on  current  management  theory  and  philos- 
ophy. No  other  types  of  comments  occurred  with  notable  frequency. 


TABLE  23.  Summary  of  Statements  Indicating  Areas  of  Needed  Improvement  in  the  ME  Course 


Number  of  Responses  by  Service* 

Totals 

Type  of  Statement 

A F 

(N=26) 

AT 

(N=35 ) 

FS 

(N=I8) 

0 

(N=181 

Nimtoer  of 
Responses 

Percent 
of  Total 

Course  Content 

15 

13 

8 

17 

53 
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Modify  Course  Approach 

7 

13 

9 

8 

37 

30 

More  practical  esphasls 

1 

4 

4 

3 

12 

10 

Other 

6 

9 

5 

5 

25 

20 

Leave  Course  As  Is 

3 

5 

2 

1 

11 

9 

Improve  Instructors 

1 

4 

2 

3 

10 

8 

Miscellaneous 

2 

5 

0 

5 

12 

10 

AT  - Airway  Facilities 
AT  - Air  Traffic 
FS  - Flight  Standards 
0 - Other 


b.  Usefulness  of  the  ME  Course.  The  ME  course  was  given  an 
overall  rating  of  at  least  moderately  useful  by  88.5  percent  of  the  graduates. 
Only  one  graduate  felt  it  had  been  of  no  value. 

2.  Evaluation  of  Course  Content. 

An  average  of  75.8  percent  of  the  respondents  rated  the  course 
topics  of  moderate  or  better  use.  This  was  the  highest  value  for  the  four 
courses  and  was  significantly  (£  < .05)^  above  the  values  for  the  PIP/PER 
and  LMR  courses. 

There  were  on  the  average  more  extreme  positive  ratings  (between  8 and  9 
per  item)  than  extreme  negative  ratings  (about  5 per  item).  The  frequencies 
of  the  positive  ratings  were  somewhat  greater  than  was  the  case  for  the  CD  and 
PIP/PER  course  and  about  the  same  as  for  the  LMR  course.  The  negative  ratings 
occurred  about  as  frequently  as  in  the  PIP/PER  and  CD  courses,  and  less  often 
than  for  the  LMR  course. 

There  were  four  topics  that  were  rated  differently  by  managers  in  the 
various  FAA  services  and  one  that  differed  as  a consequence  of  location.  These 
differences  are  discussed  below. 

a.  Team  Approach  to  Managerial  Effectiveness.  Both  of  the  topics 
taught  under  this  subject,  Concept  of  the  Team  Approach  and  Characteristics  of 
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Effective  Teamwork,  were  rated  as  at  least  moderately  useful  by  more  than  85 
percent  of  the  graduates  (Table  24). 

TABLE  24.  Usefulness  Ratings  of  HE  Course  Content 


Content  Area 

Percent 
Moderate 
or  very 

rating 
, quite, 
useful 

Team  Approach  to  Managerial  Effectiveness 

86.5 

Characteristics  of  Effective  Teaemork. 

87.1 

Concept  of  Tea*  Appraoch 

85.9 

Management  Communication 

85.7 

Role  of  Managers  In  Communication 

90.5 

Factors  In  Failure  of  Distortion  of  Communication 

84.9 

Communication  Theory 

81.5 

Management  Philosophy 

8). 3 

Croup  Influence  on  Decision  Making 

81.6 

Croup  Problem  Solving 

87.1 

Croup  Dynamics 

82.1 

Process  In  Croups 

81.8 

Content  In  Croups 

75.  3 

Management  Theory 

76.0 

Management  Style 

80.4 

Current  Theoretical  Approaches  to  Management 

71.6 

Transactional  Analysis  (TA) 

73.4 

TA  Analysis  of  Managerial  Interactions 

74.7 

TA  Theory 

72.1 

FAA  Coals  and  the  Formulation  Process 

70.7 

Development  of  Goals 

75.5 

Resources  Utilization 

74.5 

Coals  Oriented  Approach  to  Planning  and  Operations 

72.9 

Management  of  Coal  Achievement 

71.3 

DOT/FAA  Coals 

59.4 

Assessment  Process 

49.2 

Approaches  to  Assessment 

52.8 

Operating  Assessment  Centers 

45.7 

b.  Management  Communication.  This  unit  contained  the  most  highly 
rated  topic  of  all  those  in  this  course,  the  Manager's  Role  in  Communication, 
as  more  than  90  percent  of  the  graduates  felt  this  area  to  be  at  least 
moderately  useful.  The  remaining  two  areas  reviewed,  Factors  in  Communication 
Failure  and  Communication  Theory,  were  also  well  regarded  by  more  than  80 
percent  of  the  respondents. 

c.  Management  Philosophy.  This  unit  contained  only  one  topic  and 
it  was  judged  moderately  or  more  useful  by  more  than  80  percent  of  the 
respondents . 


d.  Group  Influence  on  Decision  Making.  Three  of  the  four  topics 
in  this  unit  were  rated  as  at  least  moderately  useful  by  80  percent  or  more  of 
the  graduates.  These  were  Group  Problem  Solving,  Group  Dynamics,  and  the 
Group  Process.  The  remaining  area,  Content  in  Groups,  was  judged  somewhat  less 
useful  than  the  preceding  topics  but  was  still  rated  as  moderately  useful  by 
three-fourths  of  the  graduates. 

The  responses  to  the  three  most  highly  rated  presentations  in  this  unit 
were  found  to  vary  by  chi-square  test  according  to  the  FAA  service  of  the 
respondents.  On  the  two  discussions  of  Group  Dynamics  and  Processes  in  Groups, 
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approximately  91  percent  of  the  AF  and  FS/O  groups,  but  only  66  percent  of  the 
AT  group  gave  ratings  of  at  least  moderate  usefulness  (£  < .05).  For  the 
Group  Problem  Solving  topic,  all  of  the  AF  managers  gave  ratings  of  moderate  or 
more  utility,  while  86  percent  of  the  FS/O  and  79  percent  of  the  AT  groups 
rated  the  subject  at  the  same  level  (£  < .05). 

e.  Management  Theory.  The  presentation  on  Management  Style  was 
judged  to  be  of  moderate  or  better  utility  by  80  percent  of  the  graduates. 

The  material  on  Theoretical  Approaches  to  Management  was  given  this  high  a 
rating  by  about  72  percent  of  the  respondents. 

f.  Transactional  Analysis.  Both  the  Theory  of  Transactional 
Analysis  and  Application  of  Transactional  Analysis  to  Managerial  Interactions 
were  seen  as  at  least  moderately  useful  by  approximately  three-fourths  of  the 
graduates . 

Transactional  Analysis  theory  was  seen  as  more  useful  (£  < .05)  by  those 
in  the  RO  and  FF  groups  (78.3  percent  and  74.5  percent,  respectively)  than  by 
HD  managers  (50.0  percent). 

g.  FAA  Goals  and  the  Formulation  Process.  Of  the  five  topics  in 
this  unit,  four  were  judged  to  be  at  least  moderately  useful  by  70  percent  or 
more  of  the  graduates.  The  sessions  on  Development  of  Goals  and  Resource 
Utilization  were  the  most  highly  rated  areas.  One  area,  the  presentation  on 
DOT/FAA  goals,  was  rated  substantially  below  the  others  in  usefulness,  as  only 
about  60  percent  of  the  respondents  felt  this  area  was  at  least  moderately 
useful . 

The  presentation  on  Resource  Utilization  was  rated  as  moderately  or  more 
useful  by  more  AF  managers  (96.0  percent)  than  by  FS/O  (74.3  percent)  or  AT 
(58.8  percent)  managers  according  to  chi-square  analysis  (£  < .05). 

h.  Assessment  Process.  As  noted  above,  this  unit  was  clearly  the 
least  well  regarded  of  the  course.  The  two  topics  in  this  block  were  the 
lowest  rated  in  the  course  as  just  more  than  half  the  respondents  reported 
moderate  or  better  utility  for  the  discussion  of  Approaches  to  Assessment  (53 
percent)  and  only  46  percent  felt  the  same  about  the  topic  of  Operating 
Assessment  Centers.  Both  of  these  topics  were  judged  of  no  use  by  a 
substantial  number  (13  and  17  respectively)  of  the  respondents. 

3.  The  ME  Course  and  Supervisory  Behavior. 

a.  Self-Assessment  (N  = 98). 

(1)  Most  important  changes.  When  graduates  were  asked  to 
indicate  changes  that  had  resulted  from  attending  the  ME  course  (Table  25), 

39  percent  of  the  responses  concerned  some  aspect  of  increased  knowledge  and 
better  application  of  various  management  skills  to  the  management  situation. 
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The  specific  aspect  of  such  skills  mentioned  most  often  was  improved  communi- 
cations (14  percent).  The  other  factors,  also  mentioned  with  some  frequency 
by  graduates  of  the  other  three  recurrent  courses,  were  personal  growth  (16 
percent),  improved  human  relations  (15  percent),  and  improved  general  mana- 
gerial skills  (14  percent).  On  the  less  positive  side,  another  15  percent 
felt  the  course  had  resulted  in  no  noticeable  beneficial  changes  in  their 
managerial  behavior. 


TABLE  25.  Summary  of  Statements  Concerning  Specific  Charts  in  Si4>ervlsory  Behavior 
Resulting  from  Attendance  at  l€  Course 


Nimfcer  of  Responses  by  Service* 

Totals 

Type  of  Statement 

AF  AT  FS  0 

<N*26)  (N= 35)  (N= 18 ) (Nsl8) 

Number  of  Percent 

Responses  of  Total” 

Increased  knowledge  of  specific  application  of 


managerial  programs  to  management  situations 

16 

14 

15 

12 

57 

39 

Communications 

5 

6 

4 

5 

20 

Other 

11 

8 

11 

7 

37 

25 

Personal  growth 

5 

10 

5 

3 

23 

16 

Increased  understanding  of  and  improved 
hunan  relations 

1 

9 

5 

7 

22 

15 

No  changes  reported 

6 

8 

2 

6 

22 

15 

Improved  general  managerial  skills 

8 

7 

2 

a 

21 

14 

Miscellaneous 

0 

1 

0 

2 

3 

2 

aAF  - Airway  Facilities  bSus  of  percentages  may  vary  sll^ttly  from  total  due  to  rounding  error. 

AT  - Air  Traffic 
FS  - Flight  Standards 
0 - Other 


(2)  Behavioral  ratings.  There  were  27  behavioral  items  rated 
by  the  graduates  (Table  26).  On  the  average,  67.7  percent  of  the  graduates 
felt  that  improvement  in  performance  had  resulted  from  attending  the  course. 

An  average  of  less  than  two  respondents  (1.8)  per  area  reported  diminished 
functioning. 

Several  behavioral  items  were  rated  as  improved  by  80  percent  or  more  of 
the  graduates.  These  related  to  understanding  one's  own  management  philosophy 
(#3),  understanding  interpersonal  relations  (#4),  understanding  group 
dynamics  (#6),  identifying  sources  of  conflicts  in  groups  (#7),  and  the  four 
items  concerned  with  communication  behaviors  (#13,14,15,16). 

Another  six  areas  of  performance  were  rated  improved  by  at  least  three- 
fourths  of  the  graduates.  These  were  in  the  ability  to  relate  management 
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theory  to  the  managerial  situation  (#1),  understanding  of  the  role  of  managers 
in  communications  (#16),  direction  of  subordinates  on  work  tasks  (#20),  use 
of  team  concepts  (#24),  providing  and  eliciting  feedback  (#25),  and  treating 
employees  as  individuals  (#26). 

There  were  six  performance  areas  for  which  less  than  half  the  respondents 
noted  beneficial  change.  One  of  these  was  the  application  of  the  3-D  theory 
to  day-to-day  management  (#2).  Three  of  them  concerned  goals:  understanding 
DOT/FAA  goals  (#9),  developing  appropriate  unit  goals  (#10),  and  developing 
achievement  indications  for  goals  (#11).  The  area  least  frequently  reported 
as  improved  concerned  assessment  centers.  Only  38  percent  of  the  graduates 
felt  their  understanding  of  assessment  center  concepts  (#17)  was  improved  and 
29  percent  felt  the  same  about  their  ability  to  participate  effectively  in  such 
centers  (#18). 

There  was  only  one  behavioral  assessment  on  which  the  graduates  from  the 
various  FAA  services  differed  in  their  ratings.  This  concerned  the  under- 
standing of  the  manager's  role  in  the  communication  process  withir.  the  unit 
(#16).  Chi-square  analysis  showed  that  of  those  in  the  FS/0  group,  88.9 
percent  reported  improvement,  while  75.0  percent  and  72.0  percent  of  tne  AT 
and  AF  groups  felt  improvement  had  occurred  (£  < .05). 

There  were  two  areas  where  the  ratings  differed  by  location.  Clearly, 
more  of  those  in  the  HD  and  RO  groups  (£  < .05)  felt  their  understanding  of 
assessment  center  concepts  (#17)  had  improved  (50.0  and  61.1  percent  respec- 
tively) than  did  those  in  the  FF  group  (26.1  percent).  The  ability  to 
participate  effectively  in  the  assessment  center  process  (#18)  was  judged 
improved  by  58.8  percent  of  the  RO  group,  23.9  percent  of  the  FF  group,  but 
none  of  the  HD  group  (£  < .01).  It  is  curious  that  although  several  HD 
managers  were  able  to  report  improved  understanding  of  the  assessment  center 
process,  none  felt  that  they  had  become  better  at  participating  in  the 
process . 


b.  Assessment  by  Immediate  Superiors  (N  = 60).  The  behavioral 
ratings  of  performance  improvement  by  immediate  superiors  were  slightly  lower 
than  the  changes  reported  by  the  graduates  themselves,  as  the  average  propor- 
tion noting  improved  performance  was  64.4  percent. 

The  areas  in  which  immediate  superiors  were  most  likely  to  report  improved 
performance  were  understanding  interpersonal  relations  (#4),  understanding 
group  dynamics  (#6),  understanding  the  manager's  role  in  the  communication 
process  (#16),  and  the  recognition  of  individual  needs  (#26).  For  the  areas, 
#4  and  #26,  more  than  80  percent  of  the  immediate  superiors  judged  graduates 
improved  after  MTS.  For  the  other  two  over  70  percent  of  the  immediate 
superiors  gave  the  same  high  ratings. 

The  behavioral  areas  least  likely  to  show  improvement  according  to 
immediate  superiors  were  those  concerned  with  understanding  the  assessment 
center  concept  (#17),  effective  participation  in  assessment  centers  (#18),  and 
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using  the  "team"  concept  (#24).  Between  45  and  50  percent  of  these  ratings 
reflected  improvement. 

Diminished  performance  in  graduates  was  noted  by  an  average  of  only  one 
(.96)  immediate  superior  per  area  and  never  by  more  than  three. 

There  was  one  area  that  immediate  superiors  from  the  various  FAA  services 
rated  differently.  On  the  development  of  group  action  in  problem-solving 
(#8),  88.5  percent  of  the  AT,  66.7  percent  of  the  AF , and  50.0  percent  of  the 
FS/0  superiors  reported  improvement  in  their  managers  after  the  ME  course. 
There  were  statistically  significant  differences  according  to  chi-square 
analysis  (£  < .05).  ! 

There  was  also  one  area,  use  of  the  "team  concept"  (#24)  for  which  ratings 
of  immediate  superiors  varied  as  a function  of  location  (£  < .05).  In  this 
case,  63.6  percent  of  the  immediate  superiors  from  the  RO,  50.0  percent  from 
the  FF,  and  none  of  the  HD  group  reported  improved  effectiveness. 

c.  Assessment  by  Supervisees  (N  = 82).  The  behavioral  ratings  by 
supervisees  (Table  26)  were  comparable  to  the  ratings  obtained  from  super- 
visees concerning  the  effectiveness  of  the  PIP  and  CD  courses.  An  average  of 
47.7  percent  of  the  supervisees  reported  improved  performance  in  the  managers 
who  had  attended  the  ME  course.  This  value  was  significantly  lower  than  the 
corresponding  values  obtained  by  graduates  (£  < .01)2  and  by  superiors  (£  < 
.01)2. 

In  only  the  one  area,  that  of  initiating  appropriate  changes  to  improve 
organizational  effectiveness  (#21),  did  as  many  as  60  percent  of  the  super- 
visees report  performance  improvement  in  graduates.  An  additional  seven 
behavioral  areas  were  judged  improved  by  at  least  50  percent  of  the  respond- 
ents. These  included  relations  with  employees  (#4),  developing  viable  unit 
goals  (#10),  improved  communications  (#13),  expressing  directives  and 
assignments  in  a clear  fashion  (#22),  use  of  the  team  concept  in  decision 
making  (#24),  treating  employees  as  individuals  (#26),  and  providing  oppor- 
tunities for  employee  growth  and  development  on  the  job  (#27). 

The  area  least  frequently  judged  to  show  improvement  was  concerned  with 
the  ability  to  motivate  employees  (#23)  as  38  percent  of  the  supervisees  felt 
their  supervisors  had  improved  in  this  respect.  Just  over  40  percent 
reported  improvement  in  changing  ineffective  leadership  patterns  (#5), 
identification  of  sources  of  conflict  in  groups  (#7),  and  establishing 
achievement  indicators  for  goals  (#11). 

An  average  of  almost  five  (4.7)  supervisees  reported  diminished  function- 
ing for  each  area.  The  area  with  the  greatest  number  of  such  ratings  (9) 
concerned  providing  and  eliciting  effective  feedback  (#25).  Eight  supervisees 
felt  that  graduates  were  less  effective  in  using  the  "team  concept"  in 
decision  making  (#24)  after  returning  from  the  ME  course. 
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There  were  no  differences  between  the  ratings  of  supervisees  as  a function 
of  either  FAA  service  or  location. 

d.  Comparing  the  Three  Sets  of  Ratings.  Graduates  and  immediate 
superiors  reported  about  the  same  frequency  of  change  on  the  average  (68 
percent  for  graduates,  64  percent  for  immediate  superiors).  As  was  true  for 
the  PIP/PER  and  CD  courses,  supervisees  reported  the  least  amount  of  changed 
behavior  in  graduates  (48  percent).  The  difference  between  graduates  and 
their  immediate  superiors  was  not  significant,  while  the  differences  between 
these  two  groups  and  supervisees  were  highly  significant  (£  < .01  for  each 
comparison) ^ . 

The  rankings  of  the  behaviors  by  frequency  of  observed  change  for  gradu- 
ates and  immediate  superiors  showed  a modest  relationship  (rs  = .44,  £ < .05). 
There  was  no  significant  correspondence  between  rankings  for  graduates  and 
supervisees . 

All  three  groups  frequently  reported  improvements  on  the  part  of  the 
graduate  in  the  area  of  understanding  interpersonal  relationships.  Graduates 
and  immediate  superiors  felt  understanding  and  management  of  group  dynamics 
was  often  improved;  however,  this  judgment  was  not  shared  by  supervisees  as 
this  was  an  area  where  they  were  least  likely  to  report  improvement.  Gradu- 
ates and  their  immediate  superiors  also  tended  to  report  relatively  frequent 
improvement  in  understanding  of  the  manager's  role  in  the  communication 
processes  with  organizational  units.  Graduates  and  supervisees  both  tended 
to  feel  that  improvement  was  present  in  the  manager's  ability  to  communicate 
with  others. 

The  area  where  little  improvement  was  noted  by  both  graduates  and 
immediate  superiors  was  that  of  assessment  center  operation.  This  was,  as 
was  noted  earlier,  the  area  of  the  course  felt  to  be  least  useful  by  graduates. 
Graduates  and  supervisees  agreed  in  reporting  relatively  infrequent  change  in 
establishing  achievement  indicators  for  unit  goals. 

4.  Impact  on  the  Unit. 

Graduates  of  the  ME  course  most  often  felt,  as  did  those  from  the 
other  recurrent  courses,  that  the  greatest  impact  on  the  unit  ' jS  in  improved 
interaction  with  employees  (Table  27),  as  approximately  three- fourths  of  these 
managers  reported  this  effect.  Supervisees  also  were  more  likely  to  report 
this  change  than  any  other;  however,  the  proportion  reporting  improved  inter- 
action was  just  above  one-half  of  the  supervisee  group. 

Unit  efficiency  was  judged  improved  by  nearly  two-thirds  of  the  graduates 
and  about  one-half  the  supervisees.  In  the  area  of  job  satisfaction  among 
employees,  one-half  the  graduates  but  only  a third  of  the  supervisees  noted 
improvement.  Employee  morale  was  least  likely  to  be  changed,  a pattern 
observed  in  the  other  courses  as  well. 
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TABLE  27.  Percentage  of  Graduates  and  Supervisees  Rating  Conditions  In 


Their  Unit  as  Improved  After  Attendance  of  Craduate  at  TC  Course 


Item 

Craduate 

<N=08) 

Supervisee 

(N=82) 

Unit  Efficiency 

62.5 

51.2 

Employee  lob  Satisfaction 

50.0 

36.6 

Employee  Morale 

30.9 

35.3 

Supervisor' s Interaction  With  Employees 

74.5 

52.4 

Ratings  of  deterioration  of  conditions  within  the  unit  were  not  common  as 
only  one  graduate  reported  lessened  job  satisfaction  in  employees  and  two 
reported  lowered  morale  since  attending  MTS.  For  the  supervisees,  such 
reports  were  somewhat  more  frequent  as  two  respondents  reported  poorer  inter- 
action with  the  graduate  managers,  six  felt  job  satisfaction  had  decreased, 
and  nine  indicated  that  morale  in  the  unit  had  diminished  after  the  manager's 
attendance  at  the  ME  course.  This  rating  by  nine  persons  was  the  highest 
number  of  reported  deterioration  in  conditions  for  any  of  the  four  recurrent 
courses. 

No  differences  in  responses  to  these  items  were  found  for  FAA  services  or 
locations . 

IV.  Conclusions. 

There  is  considerable  uniformity  to  the  findings  from  this  survey  of  the 
MTS  recurrent  courses.  On  the  whole,  each  course  was  seen  as  beneficial  to 
the  functioning  of  its  graduates.  This  perception  was  shared  by  the  graduates 
themselves,  by  their  immediate  superiors,  and  to  a somewhat  lesser,  but  still 
notable  degree,  by  their  supervisees.  In  other  words,  it  appears  that  the 
recurrent  training  program  is  accomplishing  its  purpose  insofar  as  that 
purpose  is  to  improve  the  perceived  behavioral  effectiveness  of  the  agency's 
supervisors  and  managers  in  the  specific  areas  covered  by  the  courses. 

Although  each  of  the  courses  is  directed  toward  a specific  area  of  super- 
visory functioning,  it  should  also  be  noted  that  many  of  the  graduates  report 
general  benefits  from  attendance.  In  particular,  these  include  personal 
growth,  a better  understanding  of  one's  self  and  others,  and  a better  sense 
of  the  general  supervisory  task. 
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While  the  evaluations  of  the  effectiveness  of  the  four  courses  are  more  j 

alike  than  different,  the  evaluation  of  the  LMR  course  diverged  in  some 
notable  ways  from  the  assessments  of  the  other  three  courses.  First,  the  LMR 
course  was  judged  to  have  the  most  behavioral  impact  of  any  of  these  courses. 

This  was  particularly  true  of  the  assessments  by  supervisees.  Second, 

although  the  course  as  a whole  was  judged  to  be  very  useful,  particularly  in 

the  area  of  grievances,  it  also  contained  material,  specifically  on  contract 

negotiations,  that  was  seen  as  having  very  little  value.  In  fact,  the  ratings 

on  the  contract  negotiation  topics  were  by  far  the  lowest  of  any  obtained  for 

all  the  courses.  Certainly  if  one  were  looking  to  modify  this  course,  this  is 

an  instructional  unit  that  should  receive  priority  attention.  Finally  it  , 

should  be  noted  that  while  the  behavioral  impact  of  this  course  was  the  most  1 

notable  of  the  four,  i.e.,  graduate  supervisors  apparently  became  more  adept 

at  handling  union-related  matters  after  attending  the  course,  the  impact  on  I 

conditions  within  the  unit  was  no  greater,  and  perhaps  somewhat  less  than  for 

the  other  courses.  The  data  oh  the  LMR  course  provide  no  clear  indications  as  j 

to  why  the  relatively  strong  improvement  in  supervisory  effectiveness  in  the  ' 

LMR  area  is  not  particularly  associated  with  similarly  improved  interpersonal 

relationship  between  supervisor  and  employees,  unit  efficiency,  job  satisfac-  j 

tion,  or  morale.  Perhaps  it  has  something  to  do  with  the  nature  of  the  union 

presence  in  the  agency;  i.e.,  that  improved  ability  to  deal  with  union  matters 

simply  has  a focused  effect  on  LMR  aspects  of  work  and  this  ability  is  not  as, 

or  at  least  no  more,  relevant  to  the  general  work  factors  mentioned  above  than 

it  is  in  the  other  courses. 

There  is  not  much  else  that  needs  to  be  said  about  the  other  three  courses 
that  is  not  already  apparent  within  the  body  of  the  findings.  The  exposure  in 
the  CD  course  to  conceptualizations  of  discipline  that  are  alternatives  to 
punishment  seems  particularly  valuable  to  supervisors.  Management  philosophy 
and  team  action  were  the  most  valued  presentations  in  the  ME  course.  The 
various  areas  within  the  PIP/PER  course  are  rated  with  a considerable  degree  of 
uniformity;  nothing  was  reported  as  strikingly  more  or  less  useful  than  any 
other  area,  with  the  one  minor  exception  of  the  discussion  on  the  relation  of 
PIP/PER  to  other  FAA  programs.  However,  even  this  area  was  viewed  as  useful 
by  more  than  half  the  respondents. 

As  for  weaknesses  in  these  courses,  the  only  specific  areas  not  already 
mentioned  that  may  be  of  questionable  value  are  those  concerned  with  assess- 
ment centers  in  the  ME  course  and  certain  aspects  of  ethical  conduct  as 
presented  in  the  CD  course.  In  a more  general  sense  there  was  feeling  by  some 
graduates  of  each  course  that  a more  practical  emphasis  is  needed  in  the 
presentation . 

The  specific  impact  of  the  courses  on  behavior  was  most  noticeable  in  a 
few  areas.  For  the  PIP/PER  course,  it  was  an  increased  understanding  of  the 
effects  of  various  supervisory/managerial  styles  on  employees  that  showed  the 
most  frequent  gain.  For  the  LMR  course,  it  seems  that  ability  to  handle 
grievances  was  most  enhanced.  The  CD  course  opened  up  alternative  approaches 
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to  discipline,  and  particularly,  enabled  graduates  to  shift  away  from  imposed 
to  self-discipline  approaches.  The  major  behavioral  contribution  of  the  ME 
course  was  probably  in  the  area  of  improved  coimunicat ions  processes. 

In  considering  the  meaning  of  any  evaluative  survey  such  as  this,  the 
question  of  the  accuracy  of  the  findings  must  be  considered.  Does  this 
evaluation  appropriately  represent  the  effectiveness  of  MTS  recurrent 
training?  Several  characteristics  of  the  data  suggest  an  affirmative  answer. 
First,  the  overall  tenor  of  the  findings  is  consistent  both  within  and 
between  courses,  between  types  of  respondents  (graduates,  immediate  superiors, 
and  supervisees),  and  between  different  aspects  of  the  FAA  organization. 
However,  within  this  overall  consistency  are  the  specific  variations  that 
suggest  that  the  judgments  provided  by  respondents  were  reasonably  considered. 
The  ratings  of  the  presentations  on  contract  negotiations  in  the  LMR  course 
are  a case  in  point.  Furthermore,  there  were  several  differences  between  the 
respondents  from  different  services  and  locations  that  suggested  that  there 
were  appropriate  variations  in  attitudes  about  certain  aspects  of  these 
courses.  Thus,  it  seems  reasonable  to  conclude  that  this  survey  provides  a 
satisfactory  estimate  of  the  reaction  of  agency  personnel  to  the  MTS 
recurrent  training  program.  It  is  clear  that  the  reaction  is  by  most  accounts 
a favorable  one. 

In  sum,  each  of  the  recurrent  training  courses  appears  to  be  providing  a 
useful  service  to  FAA  supervisors  and  managers.  While  some  modifications  to 
the  courses  may  be  in  order,  it  seems  that  the  basic  goal  of  improving  the 
performance  of  those  in  leadership  positions  within  the  agency  is  being  met 
by  this  program. 
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Appendixes 


These  appendixes  contain  the  questionnaires  used  in  the  evaluation. 
Appendix  I contains  the  questionnaires  provided  graduates  of  the  four 
courses.  Appendix  II  includes  the  questionnaires  sent  to  immediate  superiors 
of  graduates  and  Appendix  III  contains  the  questionnaires  sent  to  super- 
visees. In  each  appendix  the  cover  letters  and  demographic  forms  are  shown 
only  once. 


APPFNDIX  I 


DEPARTMENT  OF  TRANSPORTATION 

FEDERAL  AVIATION  ADMINISTRATION 

DATE 


IN  REPLY 
REFER  TO 


AAC-118 


subject  Evaluation  of  MTS  Recurrent  Training 


AERONAUTICAL  CENTER 
r o »ox  non 

OKLAHOMA  CITY.  OKLAHOMA  7»1I» 


from  Chief,  Clinical  Psycholoqy  Research  Unit,  AAC-118 
to  Participating  Supervisors 


Here  is  the  questionnaire  about  the  Management  Training  School's  (MTS) 
Recurrent  Training  Program  discussed  with  you  recently  by  phone.  Also 
enclosed  is  the  questionnaire  for  your  immediate  supervisor.  The 
questionnaire  for  your  employee  has  been  mailed  directly  to  him  or  her. 


We  appreciate  your  willingness  to  help  us  with  this  evaluation.  When 
you  have  completed  the  questionnaire,  return  the  answer  sheet  In  the 
enclosed  envelope. 

Thanks . 

C ^kO 

ROGfR  r.  SMITH,  PH.D.  ) 


3 enclosures 


MTS 

Recurrent  Course 
Questionnaire 


Instructions 


1.  Do  MOT  put  your  name  on  this  questionnaire.  We  are  interested 
in  your  frank  and  candid  responses  to  the  items;  therefore,  this 
survey  is  entirely  confidential  and  anonymous. 

2.  The  questionnaire  consists  of  rating  scales  and  a few  open-ended 
questions.  Work  quickly--use  your  first  impression  to  answer 
questions . 

3.  Use  the  enclosed  answer  sheet  to  mark  your  responses.  Fill  in  the 
space  corresponding  to  your  choice  of  answer  with  a pencil.  If 
you  do  change  your  mind,  erase  thoroughly . 

4.  When  you  are  finished,  please  place  the  questionnaire  in  the 
enclosed  postage-paid  envelope  and  mail  it  directly  to  the  address 
on  the  envelope. 

3.  If  you  have  any  questions,  please  call: 

Dr.  Roger  C.  Smith,  AAC-118 

T AA-CAMt 

P.0.  Box  2b082 

Oklahoma  City,  Oklahoma  731?3 

Telephone:  (408)  686-4846 

f TS  Dumber:  73?-484 6 
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Information  Sheet 

Age  M F CS  level  or  Wage  Grade  (WC)  level  

FAA  Program:  AF  AT  CS  Other  (specify)  

Present  position  (sector  manager,  team  supervisor,  assistant  chief,  branch  chief,  etc.) 


Present  location  (check  one): 

Washington  headquarters  

Regional  office  

NAFEC  

Aerocenter 

Field  office/facility  (specify  office/facility  type) 

Total  FAA  experience  (years  and  months) 

FAA  supervisory/managerial  experience  (years  and  months) 

Time  in  present  position  (years  and  months) 

MTS  attendance  (indicate  the  number  you  have  attended)  . 
Basic  course: 

Supervisory  (month/year)  

Managerial  (month/year)  

Refresher  course: 

Supervisory  (PIP/PER)  Recurrent  (month/year) 
Labor-Management  Relations  (month/year) 
Constructive  Discipline  (month/year) 

Management  Effectiveness  (month/year)  _ 
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PIP/PER  COURSE 
(Supervisory  Recurrent) 


1.  In  general,  how  useful  was  this  MTS  course? 


a.  very 

b.  generally 

c.  moderately 

d.  slightly 

e.  not  at  all 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  2-6. 

To  what  extent  have  conditions  in  your  unit  changed  as  a result  of 
your  attendance  at  this  MTS  course? 

a.  much  better 

b.  better 

c.  no  change 

d.  worse 

e.  much  worse 

2.  with  respect  to  efficiency 

3.  employee  job  satisfaction 
A.  employee  morale 

5.  your  interaction  with  employees 

6.  your  understanding  of  the  supervisory/managerial  role 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  7-9. 

In  general,  how  satisfied  are  you? 

a.  very  satisfied 

b.  satisfied 

c.  indifferent 

d.  dissatisfied 

e.  very  dissatisfied 

7.  working  for  the  FAA 

8.  being  a supervisor/manager 

9.  your  current  position  as  a supervisor/manager 
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PIP/PER 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  10-31. 

Please  rate  the  following  specific  course  content  areas  in  terms  of  the 
usefulness  of  each  to  you  in  your  present  position. 

a.  not  at  all  useful 

b.  slightly  useful 

c.  moderately  useful 

d.  guite  useful 

e.  extremely  useful 

If  you  do  not  remember  a subject,  leave  the  space  for  the  item  blank. 


Management  Theory 

10.  Current  theoretical  approaches  to  supervision  and  management 

11.  Management  style 

12.  Management  theory  applied  to  performance  Improvement 


Communication 

13.  Theory  and  principles  of  human  communication 

14.  Application  of  communications  principles  In  work  settings 

15.  Communication  and  managerial  style 

16.  Application  of  communication  principles  of  PIP/PER  review 
sessions 


Performance  Improvement  Program  (PIP) 

17.  The  philosophy  of  PIP 

18.  Relation  of  PIP  to  other  FAA  programs 

19.  Development  of  major  job  assignments  (MDA's) 

20.  Measuring  the  results  expected  from  employees 


Counseling 

21.  Review  of  principles  of  employee  counseling 
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PIP/PLR 


Performance  Appraisal  (PER) 

22.  Purposes  of  appraisal 

23.  Methods  of  appraisal 

24.  Problems  in  appraisal 

25.  Philosophy  of  appraisal 

26.  Establishing  performance  ratings 

27.  Relationship  of  appraisal  to  other  FAA  programs 

28.  Purposes  of  the  PER 

29.  Use  of  the  PER 

30.  Approaches  to  discussing  appraisals  with  employees 

31.  PER  appeals  process 
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PIP/PER 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  32-68. 

To  what  extent  has  your  attendance  at  this  MTS  course  changed  your  effective- 
ness In  the  areas  listed  below? 

a.  considerable  improvement 

b.  some  improvement 

c.  no  change 

d.  some  reduction 

e.  considerable  reduction 

32.  Understanding  my  own  supervisory/managerial  style 

33.  Understanding  the  effects  of  various  supervisory/managerial  styles 
on  employees 

34.  Relating  managerial  theory  to  my  current  circumstances 

35.  Understanding  my  motivations  as  an  FAA  employee  and  supervisor/manager 

36.  Understanding  the  motivation  of  employees 

37.  Apply  principles  of  communication  to  relations  with  employees 

38.  Present  information  to  employees  in  a clear  fashion 

39.  Present  information  to  employees  in  a helfpul  manner 

40.  Applying  the  principles  of  human  relations  and  communications  to 
the  PIP/PER  program 

41.  Understand  the  relationship  of  PIP  to  other  FAA  programs 

42.  Understanding  the  relation  of  major  job  assignments  (MDA's)  to  PIP 

43.  Accurate  identification  of  MGA's 

44.  Establishment  of  results  expected  for  MOA's 

45.  Effective  measurement  of  results  obtained  in  MDA's 

46.  Attend  to  employee  needs  in  supervisory/managerial  actions 

47.  Appropriate  variation  in  managerial  style  as  circumstances  warrant 

48.  Effective  conduct  of  performance  review 

49.  Understanding  relation  of  PIP  to  PER 
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PIP/PLR 


50.  Use  of  results-oriented  approach  in  performance  review 

51.  Application  of  appropriate  principles  and  techniques  to  employee 
counseling 

52.  Use  appropriate  communication  techniques  during  performance  reviews 

53.  Definition  of  objectives  of  counseling  sessions 

54.  Understand  purpose  of  employee  appraisal 

55.  Understand  effective  use  of  methods  for  appraisal 

56.  Awareness  of  FAA  policy  on  performance  appraisal 

57.  Development  of  well-defined  performance  standards 

58.  Understand  requirements  for  determining  acceptable  levels  of  competance 

59.  Understand  the  relationship  of  appraisal  to  other  FAA  programs 
(MPP , EEO,  LMR , Awards) 

60.  Understanding  of  the  PER 

61.  Ability  to  properly  prepare  PER 

62.  Effectiveness  in  review  of  PER  with  employee 

63.  Understanding  process  of  appeal  of  PER 

64.  Understand  supervisor/manager’s  role  in  appeal  process 

65.  Understand  requirements  for  denial  of  within-grade  increments 

66.  Solicitation  of  employee  involvement  in  development  of  performance 
standards 

67.  Adequacy  of  the  evaluation  of  employee  performance 

68.  Fairness  of  the  evaluation  of  employee  performance 
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Labor-Management  Relations 
( LMR) 


1.  In  general,  bow  useful  was  this  MTS  course? 


a.  very 

b.  generally 

c.  moderately 

d.  slightly 

e.  not  at  all 


USE:  THF  FOLLOWING  SCALT  FOR  ITFMS  ?-6. 

To  what  extent  have  conditions  in  your  unit  changed  as  a result  of  your 
attendance  at  this  MTS  course? 

a.  much  better 

b.  better 

c.  no  change 

d.  worse 

e.  much  worse 

?.  with  respect  to  efficiency 

3.  employee  job  satisfaction 

4.  employee  morale 

5.  your  interaction  with  employees 

6.  your  understanding  of  the  supervisory/managerial  role 

USE  THr  FOLLOWING  SCALF  FOR  ITFMS  7-9. 

In  general,  how  satisfied  arc  you? 


a. 

very  satisfied 

b. 

satisfied 

c. 

i ndi  f ferent. 

d. 

dissatisfied 

e. 

very  dissatisfied 

7.  working 

for  the 

EAA 

8.  being  a 

super visor /manager 

9.  your  current  position  as  a supervisor/manager 


"Trr 
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IIS(  rill  FOLLOWING  SCALf  LOR  I Tr NS  10-44. 

Please  rate  the  following  specific  course  content  areas  in  terms  of  the 
usefulness  of  each  to  you  In  your  present  position. 

a.  not  at  all  useful 

b.  slightly  useful 

c.  moderately  useful 

d.  quite  useful 

e.  extremely  useful 

If  you  do  not  remember  a subject,  leave  the  space  blank  for  the  item. 


background  and  Overview 

10.  fxccutive  Order  11491 

11.  The  LAA  policy  and  philosophy  of  LMR 
Mature  of  Union/Management  Relation 

l?.  Types  of  LMR  relations 

13.  Roles  of  supervisor/managers  in  LMR 

Pay  to  Pay  Operations  in  a Union  facility 

14.  Negotiating  contracts 

15.  Pealing  with  unions 

16.  Contract  admi nistration 

17.  Unfair  labor  practices  (IILP's) 

Management /l  abor  Relations  and  Communl cat ion 

18.  fmployee  rights 

19.  Relations  when  no  union  recognition  has  been  accorded 

?.0,  Relations  when  exclusive  union  recognition  has  been  accorded 
(’l.  Relations  when  rational  consultation  rights  have  been  accorded 
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Cor, tract  Negotiations 

22.  Action  prior  to  negotiations 

23.  The  negotiation  process 

24.  Post-negotiation  responsibilities 

Dealing  with  Unions  on  Contract 

25.  Written  agreement  vs.  unwritten  understandings 

26.  Roles  of  management 

27.  Roles  of  union 

Contract  Administration 

28.  Understanding  contracts 

29.  Administration  of  contracts 

Unfair  Labor  Practices  (ULP) 

30.  Understanding  IILP  actions 

31.  Procedures  for  ULP's 

Cr i evances 

32.  CSC  guidelines  for  appeals  and  grievances  systems 

33.  The  F AA  grievance  system 

34.  tlegotiated  systems 

35.  Nature  of  grievances 

36.  Union  involvement  in  grievances 

37.  Documenting  grievances 

38.  Processing  grievances 
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Arbitration 


39 

Uses  of  arbitration 

AO. 

Alternatives  to  arbitration 

41. 

Mechanics  of  arbitration 

42. 

The  supervisor/manager's  role 

in  arbitration 

43. 

The  arbitration  bearing 

44. 

Preparing  grievance  cases  for 

arbitration 
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IJSF  THF  FOLLOWING  SCALF  FOR  ITFM5  43-89. 

To  what  extent  has  your  attendance  at  this  MTS  course  changed  your 
effectiveness  in  the  areas  listed  below? 

a.  considerable  Improvement 

b.  some  improvement 

c.  no  change 

d.  some  reduction 

e.  considerable  reduction 

43.  Understanding  of  Fxecutive  Order  11491  on  LMP  in  government 

46.  Knowledge  of  the  rights  and  restrictions  detailed  in  FO  11491 

47.  Application  of  the  FAA  policy  and  philosophy  of  LMR  to  one's  own 
organizational  unit 

48.  Understand  the  meaning  of  exclusive  recognition  for  unions 

49.  Awareness  of  the  various  relationships  that  can  exist 
between  labor  and  management 

30.  Knowledge  of  the  roles  of  personnel  involved  in  the  LMR 
process  (line,  staff,  etc.) 

31.  Understanding  of  the  Impact  of  unionization  on  day  to  day  operation 
of  an  organization 

3?.  Awareness  of  communication  requirements  associated  with  the  presence 
of  labor  organizations 

33.  Awareness  of  problems  surrounding  the  presence  of  labor  orqani zat ions 

34.  Analysis  and  interpretation  of  contracts 

33.  Ability  to  execute  role  of  management  in  contract  administrat ion 

56.  Understanding  of  the  union  representative's  role  in  administering 
contracts 

57.  Knowledge  of  procedures  used  for  handling  ULP's 

38.  Knowledge  of  the  requirements  for  meetings,  conferences,  and 
consultations  with  labor  organizations 

59.  Knowledge  of  procedures  to  be  used  in  meeting  with  labor 
organizations 

60,  Knowledge  of  requirements  and  limitations  on  communication  with 
rionlabor  or  nonrecoqnlzed  labor  organizations 
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61.  Knowledge  of  FAA  policy  on  use  of  government  facilities  by  labor 
organizations 

67.  Knowledge  of  the  steps  and  actions  which  must  be  undertaken  in 
leading  to  negotiations 

63.  Awareness  of  negotiating  techniques 

64.  Awareness  of  means  available  for  resolving  differences  arising  in 
negotiations 

65.  Understanding  of  the  role  of  the  supervisor/manager  in  administering 
a labor  contract 

66.  Recognition  of  problem  areas  in  relations  with  unions  under  contract 

67.  Appreciation  of  the  role  of  the  union  representative  in  LMR 

68.  Knowledge  of  indicators  for  evaluating  the  effectiveness  of  the 
labor  contract 

69.  Contract  analysis  and  interpretation 

70.  Recognition  of  ULP's 

71.  Awareness  of  procedures  for  processing  ULP's 

72.  Ability  to  resolve  ULP's  in  a satisfactory  manner 

73.  Understand  differences  between  complaints,  grievances,  ULP's, 
and  appeals 

74.  Knowledge  of  CSC  criteria  for  grievance  systems 

75.  Awareness  of  the  FAA  grievance  system 

76.  Knowledge  of  procedures  required  in  a negotiated  grievance  system 

77.  Understanding  of  the  riqhts  of  all  parties  in  the  grievance  system 

78.  Recognition  of  grievances 

79.  Ability  to  resolve  and  process  a grievance,  includinq  the  required 
documentation 

80.  Awareness  of  supervisory /managerial  responsibility  as  to  notification 
of  union  representatives  of  grievance  actions 

81.  Knowledge  of  representation  rights  of  employees  and  union  in 
grievance  actions 
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8?.  Awareness  of  the  grievance  format 

83.  Understanding  of  the  arbitration  process 

84.  Awareness  of  available  options  prior  to  arbitration 

85.  Ability  to  recognize  and  construct  the  essential  elements  of  issues 
to  be  discussed  in  arbitration 

86.  Awareness  of  alternatives  to  arbitration 

87.  Understanding  of  arbitration  procedures 

88.  Ability  to  prepare  a grievance  case  for  arbitration 

89.  Understanding  of  the  supervisor/manager's  role  in  arbitration 
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Constructive  Discipline  Course 


1.  In  general,  how 

useful  was  this  MTS  course? 

L 

a. 

very 

b. 

generally 

c. 

moderately 

d. 

slightly 

e. 

not  at  all 

USE  THE  FOLLOWING  SCALE  FOR  ITEMS  2-6. 

To  what  extent  have  conditions  in  your  unit  changed  as  a result  of 
your  attendance  at  this  MTS  course? 

a.  much  better 

b.  better 

c.  no  change 

d.  worse 

e.  much  worse 

2.  with  respect  to  efficiency 

3.  employee  Job  satisfaction 

4.  employee  morale 

3.  your  Interaction  with  employees 

6.  your  understanding  of  the  supervisory/managerial  role 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  7-9. 

In  general,  how  satisfied  are  you? 

a.  very  satisfied 

b.  satisfied 

t c.  indifferent 

d.  dissatisfied 

e.  very  dissatisfied 

7.  working  for  the  FAA 

8.  being  a supervisor/manager 

> 9.  your  current  position  as  a supervisor/manager 

I 
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USE  THE  FOLLOWING  SCALE  FOR  ITEMS  10-35. 

Please  rate  the  following  specific  course  content  areas  in  terms  of 
the  usefulness  of  each  to  you  in  your  present  position. 

a.  not  at  all  useful 

b.  slightly  useful 

c.  moderately  useful 

d.  quite  useful 

e.  extremely  useful 

If  you  do  not  remember  a subject,  leave  the  space  blank  for  the  item. 


Human  Relations 

10.  Review  of  motivational  theory 

11.  The  supervisor ' s/manager ' s role  in  human  relations 

12.  The  impact  of  the  organizational  environment  on  employee 
behavior 


FAA  Policy  for  Discipline 

13.  The  FAA  philosophy  of  discipline 

14.  FAA  standards  of  conduct 

15.  The  role  of  the  supervisor/manager  in  the  area  of  discipline 

16.  Relation  of  discipline  to  other  FAA  programs 

Meaning  and  Purpose  of  Discipline 

17.  Concepts  of  positive  vs.  negative  discipline 

18.  Relationship  of  discipline  to  employee  motivation 

19.  The  role  of  supervisor/manager  in  preventing  the  need  for 
corrective  discipline 

20.  Identifying  potential  problem  areas 
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Informal  Discipline 

21.  Available  disciplinary  actions 

22.  How  to  employ  informal  discipline  actions 

23.  Approaches  to  the  disciplinary  interview 

Formal  Discipline 

24.  Letter  of  reprimand 

25.  Minor  adverse  actions 

26.  Major  adverse  actions 

27.  Employee  appeals  system 

Other  Behavioral  Problems 

28.  Political  activities 

29.  Financial  obligations 

30.  Alcoholic  abuse 

Ethical  Conduct 

31.  Outside  employment/firiancial  interest 

32.  Gifts,  favors,  entertainment 

?3.  Supervisor/managerial  responsibilities 

Investigations  and  Security 

34.  Investigative  and  security  resources  available  to  supervisor/managers 

35.  Supervisory/managerial  role  in  investigative  actions 
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USE  THE  FOLLOWING  SCALE  FOR  ITEMS  36-69. 

To  what  extent  hds  your  attendance  at  this  MTS  course  changed  your 
effectiveness  in  the  areas  listed  below? 

a.  considerable  improvement 

b.  some  improvement 

c.  no  change 

d.  some  reduction 

e.  considerable  reduction 

36.  Understanding  the  motivational  factors  that  relate  to  work 
performance 

37.  Understand  the  factors  influencing  the  management  style  used  by 
super visor /managers 

38.  Awareness  of  FAA  policy  and  philosophy  of  positive  discipline 

39.  Understanding  supervi sor/manager ' s responsibilities  for  discipline 

40.  Understanding  the  relation  of  discipline  to  programs  such  as 
LEO  and  LMR 

41.  Viewing  discipline  in  a positive  manner  rather  than  strictly 
punishment 

4 2.  Promoting  self-discipline  in  employees 

43.  Understanding  the  relation  of  positive  discipline  to  motivational 
factors  influencing  employee  behavior 

44.  Understanding  of  the  relation  of  discipline  to  organizational  goals 

45.  Awareness  of  the  contrast  between  positive  self-discipline  and 
negative  imposed  discipline 

46.  Implementing  supervisory /manager ial  practices  to  prevent  need  for 
corrective  (Imposed)  discipline 

47.  Identification  of  causes  of  employee  behavioral  problems 

48.  Awareness  of  methods  available  to  identify  causes  of  behavior  problems 

49.  Recognition  of  symptoms  of  problems 

50.  Awareness  of  causes  of  deteriorating  discipline 

51.  Understand  actions  to  be  taken  to  determine  causes  of  deterioration 
in  discipline 
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57 .  Knowledqe  of  informal  disciplinary  action 

53.  Understanding  of  correct  procedures  to  accomplish  informal 
disciplinary  action 

54.  Ability  to  conduct  an  effective  disciplinary  interview 

55.  Correct  use  of  the  letter  of  reprimand 

56.  Understanding  of  employee  grievance  procedures  in  relation  to 
disciplinary  action 

57.  Understanding  of  the  roles  of  the  parties  involved  in  a grievance  action 

58.  Awareness  of  documentation  and  administrative  requirements  for  a 
disciplinary  action 

59.  Knowledge  of  how  to  take  a minor  adverse  action 

60.  Awareness  of  process  for  appeal  of  minor  adverse  action 

61  Understanding  of  the  supervisor/manager' s role  in  processing  appeals 
for  minor  adverse  actions 

62.  Ability  to  prepare  a letter  of  proposal  to  take  an  adverse  action 

63.  Understanding  of  how  to  Undertake  a major  adverse  action 

64.  Understanding  of  process  for  appeal  of  a major  adverse  action 

t 

65.  Awareness  of  the  supervisor /manager ' s responsibilities  with  respect  to 
employee  behavioral  problem  in  finance,  alcohol  abuse,  and  political 
activity 

66.  Knowledge  of  the  FAA  Ethical  Conduct  Program 

67.  Awareness  of  supervisor/manager's  responsibilities  in  the  Tthical 
Conduct  Program 

68.  Awareness  of  investigative/security  resources  available 

69.  Use  of  investiqati ve/secur l ty  services 
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Managerial  Effectiveness  Course 


1.  In  general,  how  useful  was  this  MTS  course? 


a.  very 

b.  generally 

c.  moderately 

d.  slightly 

e.  not  at  all 


USE  THE  FOLLOWING  SCALE  LOR  ITEMS  2-6. 

To  what  extent  have  conditions  in  your  unit  changed  as  a result  of 
your  attendance  at  this  MTS  course? 


a.  much  better 

b.  better 

c.  no  change 

d.  worse 

e.  much  worse 

2.  with  respect  to  efficiency 

3.  employee  job  satisfaction 

4.  employee  morale 

5.  your  interaction  with  employees 

6.  your  understanding  of  the  supervisory/managerial  role 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  7-9. 

In  general,  how  satisfied  are  you? 

a.  very  satisfied 

b.  satisfied 

c.  indifferent 

d.  dissatisfied 

e.  very  dissatisfied 

7.  working  for  the  FAA 

8.  being  a supervisor/manager 

9.  your  current  position  as  a supervisor/manager 
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USE  THE  FOLLOWING  SCALE  FOR  ITEMS  10-30. 

Please  rate  the  following  specific  course  content  areas  in  terms  of 
the  usefulness  of  each  to  you  in  your  present  position. 

a.  not  at  all  useful 

b.  slightly  useful 

c.  moderately  useful 

d.  quite  useful 

e.  extremely  useful 

If  you  do  not  remember  a subject,  leave  the  space  for  the  item  blank. 
Management  Theory 

10.  Current  theoretical  approaches  to  management 

11.  Management  style 

Management  Philosophy 

12.  The  study  of  managerial  philosophy 

Group  Influence  on  Decision  Making 

13.  Group  dynamics 

14.  Process  in  groups 

15.  Content  in  groups 

16.  Group  problem  solving 

FAA  Goals  and  the  Formulation  Process 

17.  The  goals  oriented  approach  to  planning  and  operations 

18.  DOT /FAA  goals 

19.  Resources  utilization 

20.  Development  of  goals 

21.  Management  of  goal  achievement 
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Transactional  Analysis 

22.  TA  theory 

23.  TA  analysis  of  managerial  interactions 

Management  Communication 

24.  Theory  of  human  communication 

25.  Factors  in  communication  failure  and  distortion 

i 26.  Managers  role  in  communication 

Assessment  Process 

27.  Approaches  to  assessment 

28.  Operating  assessment  centers 

Team  Approach/Organization  Effectiveness 
, 29.  Concept  of  team  approach 

30.  Characteristics  of  effective  teamwork 


- 
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USE  THE  FOLLOWING  SCALE  FOR  ITEMS  31-57. 

To  what  extent  has  your  attendance  at  this  MTS  course  changed  your 
effectiveness  in  the  areas  listed  below? 

a.  considerable  improvement 

b.  some  Improvement 

c.  no  change 

d.  some  reduction 

e.  considerable  reduction 

31.  Relating  management  theory  to  your  own  supervisory/managerial  situation 

32.  Employing  concepts  such  as  Reddin' s 3-0  theory  in  your  day-to-day 
functioning  as  a supervisor/manager 

33.  Identifying  and  evaluating  your  own  personal  supervisory/managerial 
philosophy 

34.  Understanding  your  interpersonal  relations  with  your  superiors,  peers, 
and  subordinates 

35.  Identification  of  personal  leadership  patterns  that  lead  to  problems 
with  employees 

36.  Understanding  the  dynamics  that  operate  in  groups 

37.  Identification  of  sources  of  conflicts  in  groups 

38.  Development  of  group  action  in  problem  solving  and  implementation  of 
solution 

39.  Understanding  DOT/FAA  agency-wide  goals 

40.  Developing  viable  unit  goals  based  on  national  goals 

41.  Establishing  achievement  indicators  for  unit  goals 

42.  Identification  of  problem-solving  alternatives  and  strategies  in 
your  unit 

43.  Improved  managerial  communication  through  self-analysis 

44.  Understanding  emotional  and  personal  factors  which  lead  to  failures 
or  distortions  in  communications 

45.  Employing  effective  feedback  principles  in  dealing  with  employees 

46.  Understanding  your  role  as  a supervisor/manager  in  the  communication 
process  within  your  unit 
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47.  Understanding  concepts  of  assessment  centers 

48.  Can  participate  effectively  in  the  assessment  process 

49.  Effect  proper  utilization  of  resources  in  goal  achievement 

50.  Direct  subordinate's  efforts  in  initiating,  organizing, 
and  accomplishing  task 

51.  Seek  and  initiate  appropriate  changes  when  such  changes  will  result 
in  a more  effective  organization 

52.  Express  directives  and  assignments  in  a clear  and  understandable  manner 

53.  Through  use  of  communication  skills,  able  to  motivate  employees  toward 
taking  action  and/or  modifying  behavior 

54.  Use  the  "team  concept"  in  decision  making 

55.  Provide  effective  feedback  to  others  and  elicit  the  same  from  them 

56.  Treat  each  employee  as  an  individual  with  specific  and  unique  needs 

57.  Provide  opportunities  for  the  growth  and  development  of  employees 
within  the  Job 
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APPENDIX  II 

DEPARTMENT  OF  TRANSPORTATION 

FEDERAL  AVIATION  ADMINISTRATION 

DATt  AERONAUTICAL  CENTER 

a o »o»  isoai 

,,  OKLAHOMA  CUT,  OKLAHOMA  7II1S 

REF  ERPTO  AAC-118 
subject  MTS  survey 


from  Chief,  Clinical  Psychology  Research  Unit,  AAC-118 

to  Second- line  supervisors  and  managers 

One  of  your  subordinate  supervisors  has  been  asked  to  participate  in 
an  evaluation  of  the  Recurrent  Training  he  or  she  received  at  the 
FAA's  Management  Training  School  (MTS).  As  part  of  this  evaluation, 
we  would  also  greatly  appreciate  your  assessment  of  the  impact  of 
that  training  on  the  performance  of  your  subordinate. 

Enclosed  you  will  find  a brief  questionnaire  to  use  in  recording  your 
judgments.  All  that  is  needed  is  to  complete  the  questionnaire  and 
return  the  answer  sheet  in  the  enclosed  envelope. 


We  hope  you  will  be  able  to  help  us  with  this  survey  as  the  assessment 
of  the  impact  of  the  MTS  Recurrent  Training  program  would  be  incomplete 
without  the  behavioral  judgments  of  those  other  than  the  graduates 
themselves . 


Thank  you. 


tW.  t 

ROGER  A SMITH, 


PH.D. 


2 Enclosures 
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MTS 

• Questionnaire 


Instructions 


1.  Do  NOT  put  your  name  on  this  questionnaire.  We  are  interested 
in  your  frank  and  candid  responses  to  the  items;  therefore,  this 
survey  is  entirely  confidential  and  anonymous. 

2.  The  questionnaire  consists  of  rating  scales.  Work  quickly--use 
your  first  impression  to  answer  questions. 

3.  Use  the  enclosed  answer  sheet  to  mark  your  responses.  Fill  in  the 
space  corresponding  to  your  choice  of  answer  with  a pencil.  If 
you  do  change  your  mind,  erase  thoroughly. 

4.  When  you  are  finished,  please  place  the  questionnaire  in  the 
enclosed  postage-paid  envelope  and  mail  it  directly  to  the  address 
on  the  envelope. 

5.  If  you  have  any  questions,  please  call: 

Dr.  Roger  C.  Smith,  AAC-118 

FAA-CAMI 

P.0.  Box  25082 

Oklahoma  City,  Oklahoma  73125 

Telephone:  (405i  686-4846 
FTS  Number:  732-4846 
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Information  Sheet 


' U ■ L > '■  ' 

- Vis  £R  S--ELT 

Age  M F CS  level  or  Wage  Crade  (WC)  level  

FAA  Program:  AF  AT  FS  Other  (specify)  

Present  position  (sector  manager,  team  supervisor,  assistant  chief,  branch  chief,  etc.) 


Present  location  (check  one): 

Washington  headquarters  

Regional  office  

NAFEC  

Aerocenter 

Field  office/facility  (specify  off ice/facility  type). 

Total  FAA  experience  (years  and  months).  > 

FAA  supervisory /managerial  experience  (years  and  months)  . 

Time  in  present  position  (years  and  months) 

MTS  attendance  (indicate  the  number  you  have  attended)  . . 
Basic  course: 

Supervisory  (month/year)  

Managerial  (month/year)  

Refresher  course: 

Supervisory  (PIP/PER)  Recurrent  (month/year)  

Labor-Management  Relations  (month/year)  

Constructive  Discipline  (month/year)  

Management  Effectiveness  (month/year)  
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PIP/PER  Course 
(Supervisory) 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  1-36. 

To  what  extent  has  the  attendance  of  your  subordinate  at  this  MTS  course 
changed  his  or  her  effectiveness  in  the  areas  listed  below? 

a.  considerable  improvement 

b.  some  improvement 

c.  no  change 

d.  some  reduction 

e.  considerable  reduction 

If  you  don't  know,  or  have  not  observed  the  behavior  described  in  an 
item,  please  leave  it  blank. 

1.  Understanding  the  effects  of  various  supervisory  managerial  styles 
on  employees 

2.  Relates  managerial  theory  to  his  or  her  current  circumstances 

3.  Understands  his  or  her  motivations  as  an  FAA  employee  and  supervisor/ 
manager 

4.  Understanding  the  motivation  of  employees 

5.  Applying  principles  of  communication  to  relations  with  employees 

6.  Presents  information  to  employees  in  a clear  fashion 

7.  Presents  information  to  employees  in  a helpful  manner 

1 

8.  Apply  the  principles  of  human  relations  and  communications  to 
the  PIP/PER  program 

! 

9.  Understands  the  relationship  of  PIP  of  other  FAA  programs 

10.  Understands  the  relation  of  major  job  assignments  (MOA's)  to  PIP 

11.  Accurate  identification  of  MOA's 

1?.  Establishment  of  results  expected  for  MOA's 

k 

13.  Effective  measurement  of  results  obtained  in  MOA's 
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PIP/PI  It 


14.  Attends  to  employee  needs  in  supervisory/managerial  actions 

15.  Appropriate  variation  in  managerial  style  as  circumstances  warrant 

16.  Effectively  conducts  performance  review 

17.  Understands  re’ation  of  PIP  to  PER 

18.  Uses  results-orlented  approach  In  performance  review 

19.  Applies  appropriate  principles  and  techniques  to  employee  counseling 

20.  Uses  appropriate  communication  techniques  during  performance  reviews 

21.  Defines  objectives  of  counseling  sessions 

22.  Understands  purpose  of  employee  appraisal 

23.  Understands  effective  use  of  methods  for  appraisal 

24.  Aware  of  FAA  policy  on  performance  appraisal 

25.  Develops  well-defined  performance  standards 

26.  Understands  requirements  for  determining  acceptable  levels  of  competance 

27.  Understands  the  relationship  of  appraisal  to  other  FAA  programs 
(MPP,  EEO,  LMR,  Awards) 

28.  Understands  the  PER 

29.  Able  to  properly  prepare  PER 

30.  Effectively  reviews  PER  with  employee 

31.  Understands  process  of  appeal  of  PER 

32.  Understands  supervisor /manager ' s role  in  appeal  process 

33.  Understands  requirements  for  denial  of  within-grade  increments 

34.  Solicits  employee  involvement  in  development  of  performance  standards 

35.  Evaluates  employee  performance  adequately 

36.  Evaluates  employee  performance  fairly 
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Labor-Management  Relations 
(LMR) 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  1-45. 

To  what  extent  has  the  attendance  of  your  subordinate  at  this  MTS  course 
changed  his  or  her  effectiveness  in  the  areas  listed  below? 

a.  considerable  improvement 

b.  some  improvement 

c.  no  change 

d.  some  reduction 

e.  considerable  reduction 

If  you  don't  know,  or  have  not  observed  the  behavior  described  in  an  item, 
please  leave  it  blank. 

1.  Understanding  of  Executive  Order  11491  on  LMR  in  government 

2.  Knowledge  of  the  rights  and  restrictions  detailed  in  EO  11491 

3.  Application  of  the  FAA  policy  and  philosophy  of  LMR  to  his  or  her 
own  organizational  unit 

4.  Understand  the  meaning  of  exclusive  recognition  for  unions 

5.  Awareness  of  the  various  relationships  that  can  exist  between 
labor  and  management 

6.  Knowledge  of  the  roles  of  personnel  involved  in  the  LMR  process 
(line,  staff,  etc.) 

7.  Understanding  of  the  impact  of  unionization  on  day  to  day  operation 
of  an  organization 

8.  Awareness  of  communication  requirements  associated  with  the  presence 
of  labor  organizations 

9.  Awareness  of  problems  surrounding  the  presence  of  labor  organizations 

10.  Analysis  and  interpretation  of  contracts 

11.  Ability  to  execute  role  of  management  in  contract  administration 

12.  Understanding  of  the  union  representative's  role  in  administering 
contracts 
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13.  Knowledge  of  procedures  used  for  handling  ULP's 

14.  Knowledge  of  the  requirements  for  meetings,  conferences,  and 
consultations  with  labor  organizations 

15.  Knowledge  of  procedures  to  be  used  in  meeting  with  labor 
organizations 

16.  Knowledge  of  requirements  and  limitations  on  communication  with 
nonlabor  or  nonrecognized  labor  organizations 

17.  Knowledge  of  FAA  policy  on  use  of  government  facilities  by  labor 
organizations 

18.  Knowledge  of  the  steps  and  actions  which  must  be  undertaken  in 
leading  to  negotiations 

19.  Awareness  of  negotiating  techniques 

20.  Awareness  of  means  available  for  resolving  differences  arising  in 
negotiations 

21.  Understanding  of  the  role  of  the  supervisor/manager  in  administering 
a labor  contract 

22.  Recognition  of  problem  areas  in  relations  with  unions  under  contract 

23.  Appreciation  of  the  role  of  the  union  representative  in  LMR 

24.  Knowledge  of  indicators  for  evaluating  the  effectiveness  of  the 
labor  contract 

25.  Contract  analysis  and  interpretation 

26.  Recognition  of  ULP's 

27.  Awareness  of  procedures  for  processing  ULP's 

28.  Ability  to  resolve  ULP's  in  a satisfactory  manner 

29.  Understands  differences  between  complaints,  grievances,  ULP's 
and  appeals 

30.  Knowledge  of  CSC  criteria  for  grievance  systems 

31.  Awareness  of  the  FAA  grievance  system 

32.  Knowledge  of  procedures  required  in  a negotiated  grievance  system 
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33.  Understanding  of  the  rights  of  all  parties  in  the  grievance  system 

34.  Recognition  of  grievances 

35.  Ability  to  resolve  and  process  a grievance,  including  the  required 
documentation 

36.  Awareness  of  supervisory/managerial  responsibility  as  to  notification 
of  union  representatives  of  grievance  actions 

37.  Knowledge  of  representation  rights  of  employees  and  union  in  grievance 
actions 

38.  Awareness  of  the  grievance  format 

39.  Understanding  of  the  arbitration  process 

40.  Awareness  of  available  options  prior  to  arbitration 

41.  Ability  to  recognize  and  construct  the  essential  elements  of  issues 
to  be  discussed  in  arbitration 

42.  Awareness  of  alternatives  to  arbitration 

43.  Understanding  of  arbitration  procedures 

44.  Ability  to  prepare  a grievance  case  for  arbitration 

45.  Understanding  of  the  supervisor/manager's  role  in  arbitration 
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Constructive  Discipline 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  1-34. 

To  what  extent  has  the  attendance  of  your  subordinate  at  this  MTS  course 
changed  his  or  her  effectiveness  in  the  areas  listed  below? 

a.  considerable  improvement 

b.  some  improvement 

c.  no  change 

d.  some  reduction 

e.  considerable  reduction 

If  you  don't  know,  or  have  not  observed  the  behavior  described  in  an  item, 
please  leave  it  blank. 

1.  Understands  the  motivational  factors  that  relate  to  work  performance 

2.  Understands  the  factors  influencing  the  management  style  used  by 
supervisor /managers 

3.  Awareness  of  FAA  policy  and  philosophy  of  positive  discipline 

4.  Understands  supervisor/manager ' s responsibilities  for  discipline 

5.  Understands  the  relation  of  discipline  to  programs  such  as 
EEO  and  LMR 

6.  Views  discipline  in  a positive  manner  rather  than  strictly  as 
punishment 

7.  Promotes  self-discipline  in  employees 

8.  Understands  the  relation  of  positive  discipline  to  motivational 
factors  influencing  employee  behavior 

9.  Understands  the  relation  of  discipline  to  organizational  goals 

10.  Awareness  of  the  contrast  between  positive  self-discipline  and 
negative  (imposed)  discipline 

11.  Implements  supervisory/managerial  practices  to  prevent  need  for 
corrective  (imposed)  discipline 

12.  Identifies  causes  of  employee  behavioral  problems 

13.  Awareness  of  methods  available  to  identify  causes  of  behavior  problems 

14.  Recognizes  symptoms  of  problems 
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15.  Awareness  of  causes  of  deteriorating  discipline 

16.  Understands  actions  to  be  taken  to  determine  causes  of  deterioration 
in  discipline 

17.  Knows  about  informal  disciplinary  action 

18.  Understands  correct  procedures  to  accomplish  informal  disciplinary 
action 

19.  Able  to  conduct  an  effective  disciplinary  interview 

20.  Correctly  uses  the  letter  of  reprimand 

21.  Understands  employee  grievance  procedures  in  relation  to 
disciplinary  action 

22.  Understands  roles  of  the  parties  involved  in  a grievance  action 

23.  Aware  of  documentation  and  administrative  requirements  for  a 
disciplinary  action 

24.  Knows  how  to  take  a minor  adverse  action 

25.  Aware  of  process  for  appeal  of  minor  adverse  action 

26.  Understands  the  supervisor/manager's  role  in  processing  appeals 
for  minor  adverse  actions 

27.  Able  to  prepare  a letter  of  proposal  to  take  an  adverse  action 

28.  Understands  how  to  undertake  a major  adverse  action 

29.  Understands  process  for  appeal  of  a major  adverse  action 

30.  Aware  of  the  supervisor/manager ' s responsibilities  with  respect 
to  employee  behavioral  problem  in  finances,  alcohol  abuse,  and 
political  activity 

31.  Knows  the  F AA  Ethical  Conduct  Program 

32.  Aware  of  supervisor/manager's  responsibilities  in  the  Ethical 
Conduct  Program 

33.  Aware  of  investigative/security  resources  available 

34.  Uses  investigative/security  services  appropriately 
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Management  Effectiveness 


USE  THE  EOLLOWING  SCALE  EOR  ITEMS  1-25. 

To  what  extent  has  the  attendance  of  your  subordinate  at  this  MTS  course 
changed  his  or  her  effectiveness  in  the  areas  listed  below? 

a.  considerable  improvement 

b.  some  improvement 

c.  no  change 

d.  some  reduction 

e.  considerable  reduction 

If  you  don't  know,  or  have  not  observed  the  behavior  described  in  an  item, 
please  leave  it  blank. 

1.  Employing  concepts  such  as  Reddin' s 3-D  theory  in  his  or  her  day- 
to-day  functioning  as  a supervisor/manager 

2.  Understanding  interpersonal  relations  with  his  or  her  superiors, 
peers,  and  subordinates 

3.  Identification  of  personal  leadership  patterns  that  lead  to  problems 
with  employees 

k.  Understanding  the  dynamics  that  operate  in  groups 

8.  Identification  of  sources  of  conflicts  in  groups 

6.  Development  of  group  action  in  problem  solving  and  implementation  of 
solutions 

7.  Understanding  DOT/FAA  agency-wide  goals 

8.  Developing  viable  unit  goals  based  on  national  goals 

9.  Establishing  achievement  indicators  for  unit  goals 

10.  Identification  of  problem-solving  alternatives  and  strategies  in 
his  or  her  unit 

11.  Improved  managerial  communication  through  self-analysis 

12.  Understanding  emotional  and  personal  factors  which  lead  to  failures 
or  distortions  in  communications 


89 


Ill 


13.  Employing  effective  feedback  principles  in  dealing  with  employees 

14.  Understanding  the  role  of  a supervisor/manager  in  the  communication 
process  within  his  or  her  unit 

15.  Understanding  concepts  of  assessment  centers 

16.  Participates  effectively  in  the  assessment  process 

17.  Proper  utilization  of  resources  in  goal  achievement 

18.  Directing  subordinate's  efforts  in  initiating,  organizing,  and 
accomplishing  tasks 

19.  Seeks  and  initiates  appropriate  changes  when  such  changes  will  result 
in  a more  effective  organization 

20.  Expresses  directives  and  assignments  in  a clear  and  understandable  manner 

21.  Uses  communication  skills  to  motivate  employees  toward  taking  action 
and/or  modifying  behavior 

22.  Use  of  the  "team  concept"  in  decision  making 

23.  Provides  effective  feedback  to  others  and  elicits  the  same  from  them 

24.  Treats  each  employee  as  an  individual  wit!  specific  and  unigue  needs 

25.  Provides  opportunities  for  the  growth  and  development  of  employees 
within  the  job 
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APPENDIX  III 

DEPARTMENT  OF  TRANSPORTATION 

FEDERAL  AVIATION  ADMINISTRATION 

date  AERONAUTICAL  CENTER 

r o aoi  fsotz 

IN  REPLY  ..r  OKLAHOMA  CUT.  OKLAHOMA  7IIII 

REFER  TO  nnu-  l IB 

subject  MTS  survey 


from  Chief,  Clinical  Psychology  Research  Unit,  AAC-118 
to  FAA  rionsupervisory  personnel 

Your  supervisor  has  been  asked  to  participate  in  an  evaluation  of  the 
Recurrent  Training  course  he  or  she  received  at  the  FAA's  Management 
Training  School  (MTS).  As  part  of  this  evaluation,  we  would  also 
greatly  appreciate  your  assessment  of  the  impact  of  that  training  on 
the  performance  of  your  supervisor. 

Enclosed  you  will  find  two  questionnaires.  The  first  is  focused  upon 
the  results  of  MTS  training  as  you  have  observed  them.  The  second 
is  a standardized  questionnaire  that  asks  about  your  approach  to 
your  work.  All  that  is  needed  is  to  complete  the  questionnaires 
and  return  the  answer  sheets  in  the  enclosed  envelope. 

We  hope  you  will  be  able  to  help  us  with  this  survey  as  the  assessment 
of  the  impact  of  the  MTS  Recurrent  Training  program  would  be  incomplete 
without  the  assessments  of  those  other  than  the  graduates  themselves. 

C AD 

ROGER  C.  SMITH,  PH.D. 


3 Enclosures 
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MTS 

EMPLOYEE  QUESTIONNAIRE 
Instructions 


Do  NOT  put  your  name  on  this  questionnaire.  We  are  interested 
in  your  frank  and  candid  responses  to  the  items;  therefore,  this 
survey  is  entirely  confidential  and  anonymous. 

The  questionnaire  consists  of  rating  scales.  Work  quickly--use 
your  first  impression  to  answer  questions. 

Use  the  enclosed  answer  sheet  to  mark  your  responses.  Fill  in 
the  space  corresponding  to  your  choice  of  answer  with  a pencil. 
If  you  do  change  your  mind,  erase  thoroughly. 

When  you  are  finished,  please  place  the  questionnaire  in  the 
enclosed  postage-paid  envelope  and  mail  It  directly  to  the 
address  on  the  envelope. 

If  you  have  any  questions,  please  call: 

Dr.  Roger  C.  Smith,  AAC-118 

F AA-CAMI 

P.0.  Box  2308? 

Oklahoma  City,  Oklahoma  73125 

Telephone:  (405)  686-4846 
FTS  Number:  732-4846 
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TEAR  01  'T  AWH  °ET'I°N 

WITH  ANSWER  SHEET 


Information  Sheet 

Age  M F GS  level  or  Wage  Crade  (WG)  level  

FAA  Program:  AF  AT  FS  Other  (specify) 

Present  Position  (AF  technician,  flight  inspector,  air  traffic  controller,  etc.): 


Present  location  (check  one). 

Washington  headquarters  

Regional  office  . . 

NAFEC  

Aerocenter 

Field  office/facility  (specify  office/facility  type) 

Total  FAA  experience  (years  and  months) 
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PIP/PER 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  1-4. 

To  what  extent  have  conditions  in  your  unit  changed  as  a result  of  your 
supervisor's  attendance  at  the  MTS  Performance  Improvement/Performance 
Evaluation  (PIP/PER)  course? 

a.  much  better 

b.  better 

c . no  change 

d.  worse 

e . much  worse 

1.  with  respect  to  efficiency 

2.  employee  job  satisfaction 

3.  employee  morale 

4.  supervisor's  interaction  with  employees 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  5-7. 

In  general,  how  satisfied  are  you? 

a.  very  satisfied 

b.  satisfied 

c.  indifferent 

d.  dissatisfied 

e.  very  dissatisfied 

5.  working  for  the  FAA 

6.  with  your  choice  of  occupation/profession 

7.  your  current  position  in  FAA 
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USE  THE  FOLLOWING  SCALE  FOR  ITEMS  8-34. 

To  what  extent  has  your  supervisor's  attendance  at  the  PIP/PER  course  changed 
his  or  her  effectiveness  in  the  areas  listed  below? 

a.  considerable  improvement 

b.  some  improvement 

c.  no  change 

d.  some  reduction 

e.  considerable  reduction 

If  you  don't  know,  or  have  not  observed  the  supervisory  behavior  described 
in  an  item,  leave  it  blank. 

8.  His  or  her  supervisory /managerial  style 

9.  Understanding  my  motivation  as  an  employee 

10.  Relations  with  employees 

11.  Presenting  information  to  employees  in  a clear  fashion 

12.  Presenting  information  to  employees  in  a helpful  manner 

13.  Identification  of  major  job  assignments  (MJA's) 

14.  Indication  of  results  expected  for  MOA's 

15.  Measurement  of  results  obtained  in  MOA's 

16.  Attending  to  my  needs  as  an  employee  in  supervisor/managerial  actions 

17.  Varies  his  or  her  managerial  style  as  circumstances  warrant 

18.  Effectively  conducts  performance  review 

19.  Uses  a results-oriented  approach  in  performance  review 

20.  Communicates  well  during  performance  reviews 

21.  Defines  objectives  of  counseling  sessions 

22.  Clearly  states  purposes  of  employee  appraisal 

23.  Effectively  appraises  my  performance 

24.  Develops  well-defined  performance  standards 

25.  Presentation  of  requirements  for  determining  acceptable  levels  of 
competance  . 

26.  Communicates  an  understanding  of  the  relationship  of  appraisal  to 
other  FAA  programs  (MPP,  EEO,  LMR,  Awards) 
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27.  Communicates  an  understanding  of  the  PER 

28.  Properly  prepares  PER 

29.  Effectively  reviews  PER  with  me  as  an  employee 

30.  Clearly  presents  appeal  process  for  PER 

31.  Takes  proper  role  as  supervisor /manager  in  appeal  process 

32.  Solicits  employee  involvement  in  development  of  performance 
standards 

33.  Evaluation  of  my  performance  is  done  adequately  or  better 

34.  Evaluation  of  my  performance  is  done  fairly 


Labor-Management  Relations 
(LMR) 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  1-4. 

To  what  extent  have  conditions  in  your  unit  changed  as  a result  of  your 
supervisor's  attendance  at  this  MTS  course? 

a.  much  better 

b.  better 

c.  no  change 

d.  worse 

e.  much  worse 

1.  with  respect  to  efficiency 

2.  employee  job  satisfaction 

3.  employee  morale 

4.  your  supervisor's  interaction  with  employees 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  5-7. 

In  general,  how  satisfied  are  you? 

a.  very  satisfied 

b.  satisfied 

c.  indifferent 

d.  dissatisfied 

e.  very  dissatisfied 

5.  working  for  the  FAA 

6.  with  your  choice  of  occupation/profession 

7.  your  current  position  in  the  FAA 


I 
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LMR 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  8-17. 

To  what  extent  has  your  supervisor's  attendance  at  the  MTS  course  on 
Labor-Management  Relations  (LMR)  changed  his  or  her  effectiveness  in 
the  areas  listed  below? 

a.  considerable  improvement 

b . some  improvement 

c.  no  change 

d.  some  reduction 

e.  considerable  reduction 

If  you  do  not  know,  or  have  not  observed  the  supervisory  behavior  described 
in  an  item,  leave  it  blank. 

8.  Understands  of  the  impact  of  unionization  on  day  to  day  operation 
of  labor  organization 

9.  Aware  of  problems  surrounding  the  presence  of  labor  organizations 

10.  Knows  procedures  used  for  handling  ULP's 

11.  Appreciates  the  role  of  the  union  representative  in  LMR 

12.  Recognition  of  ULP's 

13.  Ability  to  resolve  ULP's  in  a satisfactory  manner 

14.  Understands  differences  between  complaints,  grievances,  ULP's 

and  appeals  , 

15.  Understands  the  rights  of  all  parties  in  the  grievance  system 

16.  Recognition  of  grievances 


17,  Ability  to  resolve  and  process  a grievance,  including  the  required 
documentation 


Constructive  Discipline  Course 
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USE  THE  FOLLOWING  SCALE  FOR  ITEMS  1-4. 

To  what  extent  have  conditions  in  your  unit  changed  as  a result  of  you 
supervisor's  attendance  at  the  MTS  Constructive  Discipline  Course?  \ 

a.  much  better 

b.  better 

c.  no  change  ■ 

d.  worse 

e.  much  worse 

1.  with  respect  to  efficiency  i 

2.  employee  Job  satisfaction  ^ 

3.  employee  morale  i 

4.  supervisor's  interaction  with  employees 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  5-7. 

In  general,  how  satisfied  are  you? 

a.  very  satisfied 

b.  satisfied 

c.  indifferent 

d.  dissatisfied 

e.  very  dissatisfied 

5.  working  for  the  FAA 

6.  with  your  choice  of  occupation 

7.  your  current  position  as  a supervisor/manager 
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USE  THE  FOLLOWING  SCALE  FOR  ITEMS  8-19. 

To  what  extent  has  your  supervisor's  attendance  at  the  Constructive  Discipline 
Course  changed  his  or  her  effectiveness  in  the  areas  listed  below? 

a.  considerable  Improvement 

b . some  improvement 

c.  no  change 

d.  some  reduction 

e.  considerable  reduction 

If  you  don't  know,  or  have  not  observed  the  supervisory  behavior  described 
in  an  item,  leave  It  blank. 

8.  Understanding  my  motivation  as  an  employee 

9.  Views  discipline  in  a positive  manner  rather  than  strictly 
punishment 

10.  Promotes  self-discipline  in  employees 

11.  Relates  discipline  to  organizational  goals 

12.  Encourages  positive  self-discipline  as  opposed  to  negative  (imposed) 
discipline 

13.  Implements  supervisory /managerial  practices  to  prevent  need  for 
corrective  (imposed)  discipline 

14.  Can  identify  causes  of  employee  behavioral  problems 

15.  Recognizes  symptoms  of  problems 

16.  Is  aware  of  causes  of  deteriorating  discipline 

17.  Acts  appropropriately  to  determine  causes  of  deterioration  in  discipline 

18.  Effectively  employees  informal  disciplinary  action 

19.  Conducts  an  effective  disciplinary  interview 
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Managerial  Effectiveness  Course 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  1-4. 

To  what  extent  have  conditions  in  your  unit  changed  as  a result  of  your 
supervisor's  attendance  at  the  MTS  course  in  Managerial  Effectiveness? 

a.  much  better 

b.  better 

c.  no  change 

d.  worse 

e.  much  worse 

1.  with  respect  to  efficiency 

2.  employee  job  satisfaction 

3.  employee  morale 

4.  supervisor's  action  with  employees 


USE  THE  FOLLOWING  SCALE  FOR  ITEMS  5-7. 

In  general,  how  satisfied  are  you? 

a.  very  satisfied 

b.  satisfied 

c.  indifferent 

d.  dissatisfied 

e.  very  dissatisfied 

5.  working  for  the  FAA 

6.  with  your  choice  of  occupational/profession 

7.  your  current  position  in  FAA 
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USE  THE  FOLLOWING  SCALE  FOR  ITEMS  8-25. 

To  what  extent  has  your  supervisor's  attendance  at  the  Managerial  Effectiveness 
course  changed  his  or  her  effectiveness  in  the  areas  listed  below? 

a.  considerable  improvement 

b.  some  improvement 

c.  no  change 

d.  some  reduction 

e.  considerable  reduction 

8.  Relations  with  employees 

9.  Changing  personal  leadership  patterns  that  lead  to  problems  with 
employees 

10.  Identifies  sources  of  conflicts  in  groups 

11.  Develops  group  action  in  problem  solving  and  implementation  of  solution 

12.  Develops  viable  unit  goals 

13.  Established  achievement  indicators  for  unit  goals 

14.  Identifies  problem-solving  alternatives  and  strategies  in  your  unit 

15.  Has  improved  managerial  communication 

16.  Understands  emotional  and  personal  factors  which  lead  to  failures 
or  distortions  in  communications 

17.  Employs  effective  feedback  principles  in  dealing  with  employees 

18.  Directs  efforts  in  initiating,  organizing,  and  accomplishing  tasks 

19.  Seeks  and  initiates  appropriate  changes  when  such  changes  will  result 
in  a more  effective  organization 

20.  Expresses  directives  and  assignments  in  a clear  and  understandable  manner 

21.  Through  use  of  communication  skills,  is  able  to  motivate  employees 
toward  taking  action  and/or  modifying  behavior 

22.  Uses  the  "team  concept"  in  decision  making 

23.  Provides  effective  feedback  to  others  and  elicits  the  same  from  them 

24.  Treats  each  employee  as  an  individual  with  specific  and  unique  needs 

25.  Provides  opportunities  for  the  growth  and  development  of  employees 
within  the  job 
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